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I.  INTRODUCTION 


During  the  1980s  and  1990s,  the  Department  of  Defense  (DoD)  was  facing  a 
dilemma  with  regard  to  its  technology  management.  One  key  issue  was  the  time  between 
project  conception  and  utilization.  One  solution  would  find  its  beginnings  in  the 
recommendations  of  the  Packard  Commission  of  1986  [President’s  Blue  Ribbon 
Commission  on  Defense  Management,  Jun  1986]  and  the  Defense  Science  Board  reports 
of  1987,  1990  and  1991  [Reports  of  the  Defense  Science  Board,  1987,  1990,  1991].  The 
Packard  Commission  identified  a  concept  that  could  be  broadly  defined  as  a  fieldable 
sample  to  bridge  the  gap  between  prototypes  and  operational  units.  The  commission’s 
view  was  summed  up  as  “Operational  tests  should  be  combined  with  developmental  tests 
of  the  prototype  to  uncover  operational  as  well  as  technical  deficiencies  before  a  decision 
is  made  to  proceed  with  full-scale  development  [citation].”  In  this  statement  (and  similar 
complementary  conjectures)  was  the  catalyst  for  Advanced  Concept  Technology 
Demonstrators  (ACTD).  To  date,  167  programs  have  been  designated  as  ACTDs. 

This  project  is  broadly  defined  to  analyze  the  ACTD  process  and  associated  issues 
involved  within  the  process.  Our  focus  is  on  the  predictive  value  of  "management  plans" 
for  ACTDs.  We  analyzed  the  management  plans  for  19  programs  that  have  successfully 
navigated  the  ACTD  process  towards  an  attempted  acquisition  transition  and  19  that 
failed  to  transition  into  the  acquisition  process.  We  used  multivariate  regression  to 
examine  what  factors  predict  a  successful  transition,  or  not.  In  summary  we  found  the 
following  variables  were  significant  predictors. 

1 .  Budget  matches  the  schedule 

2.  Technology  Maturity 

3.  Risk  Assessment 

After  laying  out  our  analysis  we  discuss  our  findings  and  suggest  managerial 
rec  ommendations . 
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A. 


BACKGROUND 


As  defined  in  Department  of  Defense  Directive  (DoDD)  5000.1  the  primary 
objective  of  Defense  acquisition  is  to  acquire  quality  products  that  satisfy  user  needs  with 
measurable  improvements  to  mission  accomplishment  and  operational  support,  in  a 
timely  manner,  and  at  a  fair  and  reasonable  price.  [DoDD  5000.1,  January  2001] 
Department  of  Defense  Instruction  (DoDI)  5000.2  identifies  technology  transition 
mechanisms  designed  to  ensure  the  transformation  of  innovative  concepts  and  superior 
technology  to  the  user  and  acquisition  customer  through:  1)  Advanced  Technology 
Demonstration  (ATD)  programs,  2)  ACTD  programs,  and  3)  Experiments.  [DoDI  5000.2, 
January  2001] 

The  fonnal  acquisition  process,  as  directed  by  DoDD  5000.1  and  DoDI  5000.2,  is 
the  primary  mechanism  for  the  procurement  of  new  systems  and  the  introduction  of  new 
capabilities  via  new  or  upgraded  systems.  Recently,  it  has  been  recognized  that  the 
ACTD  process,  as  a  pre-acquisition  event,  provides  an  important  mechanism  and 
opportunity  for  the  war  fighter  to  try  out  and  evaluate  proposed  technology  solutions  to 
urgent  military  needs.  [ACTD  Introduction,  September  2001]  Each  ACTD  is  aimed  at 
one  or  more  war  fighting  objective  and  is  reviewed  by  the  Services,  Defense  Agencies 
and  the  Joint  Staff.  Key  criteria  by  which  ACTD  candidates  are  evaluated  consist  of:  1) 
Response  to  user  needs,  2)  Exploit  of  mature  technologies,  and  3)  Potential  effectiveness. 
[ACTD  Guidelines  -  Introduction  to  ACTDs,  May  2001]  An  ACTD  is  designed  to 
provide  a  sound  assessment  of  the  military  utility  of  a  proposed  solution  prior  to  a 
decision  on  formal  development  or  acquisition.  The  purchase  of  additional  capability 
beyond  the  residuals  provided  by  the  ACTD,  where  appropriate,  is  accomplished  through 
a  formal  acquisition  program. 

While  identified  as  tools  to  rapidly  transition  technology  to  the  war  fighter,  it  is 
not  certain  whether  ACTDs  live  up  to  their  expectations.  As  defined  in  greater  detail  later 
in  this  thesis,  ACTDs  are  two  to  four  year  programs  that,  if  successful,  may  be 
transitioned  to  the  war  fighter  as  residual  assets,  for  two  or  more  years,  or  as  a  new 
acquisition  program.  Utilization  of  residual  assets  alone  typically  lack  the  logistics  chain 
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associated  with  standard  DoD  program,  thus  limiting  useful  life.  The  acquisition 
transition  process  however,  currently  requires  funding,  via  the  Program  Objectives 
Memorandum  (POM)  cycle,  along  with  the  appropriate  DoD  acquisition  related 
documentation  to  be  available/completed  before  the  process  can  move  forward.  These 
combined  events  impart  a  two-year  acquisition  transition  window  following  the 
successful  completion  of  an  ACTD  program,  which  adversely  impacts  program 
momentum.  Additional  momentum  impacts  include  changes  in  user  organizations, 
sponsor  organizations  or  lead  service  organizations. 

The  ACTD  process  has  a  significant  level  of  management  oversight,  however 
each  program  is  highly  tailored  and  there  is  a  much  less  formal  structure  than  with  the 
standard  acquisition  process.  The  standard  process  typically  involves  programs  with 
higher  funding  levels,  which  are  therefore  governed  by  laws  and  regulations,  which  have 
to  be  addressed  by  major  defense  acquisition  programs.  For  those  ACTDs  that 
demonstrate  strong  military  utility,  the  intent  is  to  transition  into  the  formal  acquisition 
process  to  acquire  the  system  in  sufficient  quantity  to  meet  the  operational  requirement. 
However,  without  careful  preparation,  the  transition  may  result  in  the  loss  of  some  of  the 
benefits  of  the  ACTD.  For  example,  without  suitable  preparation  in  areas  such  as 
contracting,  costly  delays  -  including  a  break  in  a  production  line  -  could  occur.  Upfront 
planning  is  crucial  to  ensuring  successful  transition  of  an  ACTD  to  the  acquisition 
process.  Potential  outcomes  that  could  be  expected  depending  on  the  amount  of 
groundwork  performed  could  include: 

1 .  ACTD  does  not  transition  because  it  is  judged  to  lack  military  utility. 

2.  ACTD  does  not  transition  because  of  poor  management  (or  other 
problems). 

3.  ACTD  transitions,  but  has  problems  (due  to  poor  management,  etc). 

4.  ACTD  transitions  with  no  problems. 

The  ACTD  process  appears  to  be  performing  its  job  well,  43  out  of  98  ACTDs 
have  successfully  completed  the  demonstration  phase  based  on  DoD  statistics.  However, 
ACTD  transition  to  a  DoD  5000  series  acquisition  project  remains  a  hurdle  with  only  32 
out  of  98  ACTDs  currently  being  executed  as  acquisition  programs.  [Joint  War  fighting 
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Science  and  Technology  Plan,  February  2002].  Tailoring  of  this  process  or  defining 
transition  needs  to  benefit  the  program  and  the  war  fighter  is  required.  This  thesis  will 
attempt  to  define  those  elements  that  have  helped  or  hindered  ACTD  transitions  and 
establish  guidelines  to  assist  transitions  in  the  future. 

B.  PURPOSE 

The  purpose  of  this  project  is  to  help  improve  ACTD  program  management  by 
identifying  attributes  which  predict  transition  into  the  acquisition  process.  These 
attributes  can  furthermore  be  used  to  identify  potential  pitfalls  in  existing  ACTD 
programs. 

C.  RESEARCH  QUESTIONS 

The  research  questions  associated  with  this  project  consists  of: 

Primary 

1 .  What  are  the  most  important  variables  that  will  affect  success  or  failure  of 
an  ACTD  to  transition  into  the  nonnal  acquisition  process? 

Secondary 

1.  Were  all  managers  identified  by  name?  What  effect  does  the  inclusion  or 
exclusion  of  managers  identified  by  name  have  on  an  ACTD’s  success? 

2.  Does  the  funding  profile  correspond  to  the  development  schedule?  What 
effect  does  the  budget  profile  matched  to  a  program’s  development 
schedule  have  on  an  ACTD’s  success? 

3.  Was  a  well-defined  military  need  included  in  the  ACTD’s  management 
plan?  What  effect  does  the  inclusion  or  exclusion  of  a  thoughtful  military 
need  have  on  an  ACTD’s  success? 

4.  How  mature  was  the  technology  (especially,  software  technology)  noted 
for  inclusion  in  the  ACTD  program?  What  degree  of  commercial/off-the- 
shelf  versus  new  development  was  utilized?  What  effect  does  the  maturity 
of  a  program’s  technology  have  on  an  ACTD’s  success? 

5.  Was  a  detailed  transition  strategy  included  in  the  management  plan?  What 
effect  does  the  inclusion  or  exclusion  of  a  detailed  transition  strategy  have 
on  an  ACTD’s  success? 
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6.  Did  the  program’s  development  fall  within  the  prescribed  development 
timeline  of  two  to  four  years?  What  effect  does  a  development  schedule 
contained  to  two  to  four  years  have  on  an  ACTD’s  success? 

7.  How  in-depth  was  the  program’s  management  plan?  What  effect  does  the 
depth  of  a  management  plan  have  on  an  ACTD’s  success? 

8.  How  many  parties  were  involved  in  management  oversight  of  the 
program?  Do  fewer  or  more  parties  have  an  impact  on  an  ACTD’s 
success? 

9.  How  much  capital  investment  was  required  as  laid  out  in  the  management 
plan?  Does  the  degree  of  capital  requirement  have  an  effect  on  an 
ACTD’s  success? 

10.  How  complex  is  the  ACTD’s  technology?  To  what  degree  does  the 
interrelationship  between  technology  efforts  determine  technology 
complexity?  What  effect  does  technology  complexity  have  on  an  ACTD’s 
success? 

11.  How  risky  is  an  ACTD  effort?  To  what  degree  do  all  programmatic 
efforts  analyzed  holistically  have  on  a  program’s  risk?  What  effect  does 
assessed  risk  have  on  an  ACTD’s  success? 

D.  BENEFITS  OF  STUDY 

This  thesis  is  intended  to  define  attributes  that  involves  the  ability  of  ACTD 
programs  to  transition  to  the  acquisition  process.  Its  findings  may  have  value  by 
suggestion  how  ACTD  management  may  be  improved. 

E.  SCOPE  AND  METHODOLOGY 

The  scope  of  this  thesis  is  to:  (1)  review  of  the  ACTD  process  as  defined  under 
DoD  5000  documentation,  (2)  review  past  ACTD  programs,  (3)  perform  an  analysis  of 
multiple  ACTD  program  management  plans,  (4)  investigate  potential  areas  affecting 
ACTD  successes,  and  (5)  define  potential  pitfall  identification  which  would  aid  in  ACTD 
transition  to  acquisition  programs.  This  thesis  is  intended  to  identify  failings  in  the 
existing  ACTD  transition  process  and  conclude  with  recommended  improvements  that 
enable  a  more  stable  and  rapid  introduction  of  technology  to  the  war  fighter  through  the 
acquisition  process. 

The  methodology  used  in  this  thesis  research  consists  of  coding  programmatic 
elements  and  statistical  analysis 
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F. 


ORGANIZATION  OF  STUDY 


This  thesis  is  organized  into  four  primary  sections,  the  first  of  which  discusses  the 
acquisition  reform  revolution.  This  section  will  describe  the  ACTD  process  that  is  in 
place  today.  The  second  section  will  review  the  transition  process  and  associated  risk 
involved  with  the  ACTD  process.  The  content  will  consist  of  the  transition  plan, 
contracting  strategy,  and  the  transition  funding  along  with  the  risk  involved.  These 
sections  will  be  followed  up  by  an  analysis  of  38  randomly  chosen  ACTD  programs  that 
were  measured  against  the  research  questions.  These  38  programs  were  run  in  a  multiple 
regression  and  correlation  analysis  to  determine  what  variables  affect  the  success  or 
failure  of  an  ACTD  to  transition  into  the  nonnal  acquisition  process.  We  identified 
several  statistically  significant  variables.  The  final  section  concludes  with 
recommendations  about  how  to  improve  the  ACTD  process  in  the  future. 
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II.  THE  ACTD  PROCESS 


ACTDs  were  first  introduced  in  1995  with  twelve  authorized  demonstrations. 
These  programs  started  the  revolution  towards  a  more  rapid  acquisition  process  that  is 
still  ongoing  today.  This  process  allows  the  U.S.  government  to  reduce  its  cycle  time  and 
speed  its  delivery  of  advanced  capabilities  to  the  war  fighters  (Aldridge,  April  12,  2002). 

Chapter  II  will  discuss  the  evolution  of  the  ACTD  process;  the  initiation, 
developments  implemented  in  the  ACTD  lifespan,  the  process  and  goals  associated  with 
the  ACTDs,  and  will  close  with  a  brief  review  of  the  ACTD  programs  initiated  between 
FY95  and  FY07.  This  information  will  help  the  reader  understand  and  appreciate  how 
ACTD  programs  are  integrated  into  the  acquisition  system. 

A.  ACTD  BEGINNINGS 

In  1986,  Advanced  Concept  Technology  Demonstrations  were  identified  by  the 
Packard  Commission  as  an  idea  without  a  name.  They  presented  this  idea  of  improving 
DoD  management  and  organization  in  a  Presidential  report  (Packard,  1986).  The  report 
dealt  with  improvements  to  the  defense  acquisition  system  and  identified  several  means 
to  obtain  acquisition  reform.  Their  findings  were  critical  elements  for  the  process 
changes  that  followed.  The  commission  believed  that  through  the  use  of  demonstration 
platforms,  or  prototypes,  the  government  could  streamline  procurement  practices  to 
reduce  costs  while  at  the  same  time  gain  a  realistic  assessment  of  operational  suitability 
(South,  2003). 

Secretary  of  Defense  Dick  Cheney  took  this  idea  from  the  Presidential  report  of 
1986  and  defined  a  new  acquisition  strategy  in  his  1992  annual  report  to  the  President 
and  the  Congress  (Cheney,  1992).  The  use  of  demonstration  platfonns  instead  of  the 
traditional  production  programs  allowed  the  military  to  validate  new  concepts.  This 
would  reduce  procurement  timelines  so  Cheney  recommended  that  proven  subsystems  or 
technologies  be  inserted  into  existing  weapons  platforms  (Cheney,  1992). 
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During  that  same  time  Representative  Les  Aspin  grabbed  Cheney's  idea  and 
developed  a  "Rollover-Plus"  plan  (Aspin,  1992). 

We  would  not  commit  to  quantity  production  at  the  outset  of  the 
development.  Instead,  a  prototype  would  not  be  brought  into  full-scale 
production  until  the  resulting  component  or  system  met  stringent  criteria. 

Those  criteria  are  a)  the  technology  works,  b)  it  is  required  by 
development  of  the  threat,  or  c)  represents  a  breakthrough  that  would  alter 
battlefield  operations.  If  the  resulting  prototype  did  not  meet  those 
criteria,  however,  we  would  "rollover"  the  new  technologies  and  lessons 
learned  from  development  into  a  further  iteration  of  engineering, 
development,  and  prototyping.  (Aspin,  1992). 

Secretary  Cheney  finally  gave  this  concept  a  name  in  his  1993  annual  report.  He 
came  up  with  guidelines  for  Advanced  Technology  Demonstration  (ATD)  that  allowed 
an  ATD  to  transition  into  production  (Cheney,  1993).  His  approach  on  the  ATD  is  that  it 
could  not  stand  alone.  It  will  need  exercises  and  simulations  to  prove  the  technology  is 
ready,  manufacturing  process  is  available,  and  operations  are  understood  before  the  ATD 
is  considered.  Each  ATD  is  required  to  demonstrate  to  decision  makers  that  the 
technology  is  feasible,  affordable,  and  compatible  with  operational  concepts  (Cheney, 
1993).  The  intent  of  the  process  was  to  provide  realistic  demonstrations  of  the 
technology  development  and  involve  the  war  fighter  in  the  evaluation  process  prior  to 
commitment  of  funds  (Cheney,  1993). 

In  January  of  1994,  Les  Aspin  took  over  as  Secretary  of  Defense.  In  his  first 
annual  report  to  the  President  he  referred  to  the  ATD  as  it  became  known  from  Secretary 
Cheney  to  Advanced  Concept  Technology  Demonstration  (ACTD).  Secretary  Aspin 
went  on  to  say  that  ACTD's  need  to  have  an  integrated  effort  between  operational  users 
and  Science  and  Technology  (S&T)  community.  Where  the  user  provide  the  operational 
context,  concept  of  operations,  and  manages  the  operational  aspects  of  the  demonstration; 
while  the  S&T  community  provides  the  advanced  technology  elements  (Aspin,  1994). 
The  point  of  this  is  to  address  operational  utility  and  have  a  cost  effective  program  with 
minimal  risk  involved.  By  refining  the  operational  requirements  and  concept  design  will 
allow  the  new  capability  to  enter  the  formal  acquisition  process  with  minimal  delay  and 
cost  (Aspin,  1994). 
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Aspins  ACTD  approach  emphasized  cooperation  between  the  war  tighter  and  the 
S&T  communities.  The  ACTD  concept  would  provide  the  traditional  role  of  technical 
and  cost  risk  reduction,  while  also  providing  a  way  for  refining  the  operational  concept. 
Concurrent  with  these  efforts,  Aspin  created  the  position  of  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology)  (DUSD(AT))  to  effectively  manage  the  ACTDs  (South, 
2003). 

With  the  base  line  for  ACTDs  in  place,  Secretary  of  Defense  William  Perry 
continued  to  add  to  the  process.  His  change  allowed  the  war  fighter  to  modify  the  ACTD 
as  it  evolved  through  the  process  of  fielding  and  operational  testing  (Deutch,  1994).  An 
important  aspect  for  the  user  of  the  ACTD  is  that  it  has  operational  capability  for 
continued  use.  This  allows  the  combat  commander  flexibility  to  refine  the  doctrine  and 
tactics  to  maximize  the  technologies  capabilities.  (Deutch,  1994). 

Perry  continued  to  improve  the  ACTDs  and  made  them  official  in  his  1995  annual 
report.  Most  of  the  earlier  ideas  for  ACTDs  were  still  in  place  including  early 
involvement  by  users,  refinement  of  operational  concepts,  fielding,  and  quick  transition 
to  the  field.  In  his  annual  report  he  also  introduced  the  following  four  criteria  for  an 
ACTD  to  be  considered  (Perry,  1995). 

1 .  Offers  a  potential  solution  to  a  military  problem  or  introduces  a  significant 
new  capability.  The  Joint  Requirements  Oversight  Council  (JROC)  and 
the  unified  commanders  must  approve  and  ACTD. 

2.  Is  relatively  mature  and  contributes  to  solving  the  problem. 

3.  Has  and  executable  program  and  management  plan. 

4.  Is  a  two  to  four  year  program  that  can  be  supported  for  two  years  in  the 
field? 

The  1995  annual  report  also  defined  outcomes  for  ACTDs.  If  unsuccessful, 
ACTDs  were  to  be  terminated  or  shelved  for  future  restructuring.  Upon  the  user's 
recommendation,  an  ACTD  could  be  directly  fielded  with  minor  modifications  or  enter 
the  formal  acquisition  process  at  and  advanced  milestone  (MS  B  or  MS  C)  (South,  2003). 
Although  he  did  make  it  clear  that  the  ACTD  process  was  not  a  substitute  for  the  formal 
acquisition  process  and  was  not  to  be  used  as  a  vehicle  to  purchase  large,  complex 
weapon  systems  such  as  ships  and  tanks.  It  was  also  not  intended  to  support  acquisitions 
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of  new  systems  such  as  vehicles  or  munitions.  With  these  restrictions  Secretary  Perry 
prevented  the  ACTD  programs  from  directly  fielding  expensive  defense  acquisition 
programs. 

In  1995,  the  first  twelve  ACTD  programs  were  selected.  Because  these  programs 
are  not  followed  with  the  same  oversight  as  typical  acquisition  programs  they  have 
collected  the  interest  of  political  officials.  In  1997,  the  Office  of  the  Inspector  General 
(OIG)  audited  the  ACTD  process  (South,  2003).  The  major  topics  the  OIG  wanted  to 
look  at  were  1)  the  criteria  used  to  select  current  and  pending  ACTD  efforts,  2)  the 
process  for  detennining  the  program's  effectiveness,  and  3)  the  transition  of  the  program 
into  the  defense  acquisition  cycle.  They  also  evaluated  4)  the  adequacy  of  the  DoD 
management  controls  as  they  applied  to  the  audit  objective  (Office  of  the  Inspector 
General,  1997).  OIG  investigated  9  of  the  22  ACTDs  approved  in  FY95  and  FY96. 
They  found  five  were  questionable  choices  based  on  their  interpretation  of  the  ACTD 
selection  criteria  (South,  2003). 

OIG  also  found  four  projects  did  not  have  mature  technology.  All  four  ACTDs 
relied  significantly  on  modeling  and  simulations  because  supporting  programs  were  not 
mature.  A  recommendation  of  a  clear  and  consistent  criteria  selection  for  mature 
technologies  to  be  established  (South,  2003).  They  also  reported  that  eight  of  the  nine 
programs  assessed  did  not  have  a  declared  urgent  military  need.  OIG  recognized  that 
military  need  may  be  declared  by  certain  DoD  officials  but  indicated  that  DUSD(AT)  had 
not  defined  what  constitutes  an  urgent  military  need  or  who  may  declare  the  urgent  need 
for  the  ACTD  candidates  (South,  2003).  OIG  recommended  that  a  critical  military  need 
be  defined. 

In  1998,  General  Accounting  Office  (GAO)  received  a  request  from  the  House  of 
Representatives  to  take  a  closer  look  at  the  ACTD  process.  Specifically  they  wanted 
GAO  to  determine: 

1.  Whether  the  selection  process  included  criteria  that  were  adequate  to 
ensure  that  only  mature  technologies  were  selected  for  ACTD  prototypes. 

2.  Whether  guidance  on  transitioning  to  the  normal  acquisition  process 
ensured  that  prototypes  appropriately  completed  product  and  concept 
development  and  testing  before  entering  production. 
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3.  Whether  DoD  was  procuring  more  ACTD  prototypes  than  needed  to 
assess  the  military  utility  of  mature  technology  (Rodrigues,  1998). 

What  GAO  found  was  a  great  need  for  improvement  within  the  ACTD  process. 
GAO  reported  back  to  the  House  of  Representatives  that: 

1.  DoD's  process  for  selecting  ACTD  candidates  did  not  include  adequate 
criteria  for  assessing  the  maturity  of  the  proposed  technology  resulting  in 
the  approval  of  ACTD  projects  that  included  immature  technology. 

2.  Guidance  on  entering  technologies  into  the  normal  acquisition  process  was 
not  sufficient  to  ensure  that  prototype  completed  product  and  concept 
development  and  testing  before  entering  production. 

3.  DoD's  practice  of  procuring  prototypes  beyond  those  needed  for  the  basic 
ACTD  demonstration  and  before  completing  product  and  concept 
development  and  testing  was  unnecessarily  risky  (Rodrigues,  1998). 

With  all  these  findings  the  GAO  stated  three  recommendations  for  the  Secretary 
of  Defense  to  take  action  on  in  order  to  clarify  the  ACTD  process: 

1 .  Ensure  the  use  of  mature  technology  with  few,  if  any,  exceptions 

2.  Describe  when  transition  to  the  development  phase  of  the  acquisition  cycle 
is  necessary  and  the  types  of  development  activity  that  may  be 
appropriate. 

3.  Limit  the  number  of  prototypes  to  be  procured  to  the  quantities  needed  for 
early  user  demonstrations  of  mature  technology  until  the  items  product 
and  concept  development  and  testing  has  been  completed  (Rodrigues, 
1998). 

One  way  to  approach  the  maturity  level  issue  would  be  to  establish  a  criteria 
based  of  the  DoD  5000.2-R  manual  that  list  various  Technology  Readiness  Levels  (TRL) 
which  includes  a  description  of  what  each  of  the  levels  mean  including  examples.  In 
2002  a  gentleman  named  Jim  Sheldon  expanded  the  chart  to  include  component/system 
levels,  equipment  necessary  to  demonstrate  capability  and  potential  operation 
environment.  Based  on  the  TRL  chart,  level  1  through  4  would  not  be  acceptable  levels 
for  ACTDs.  TRLs  5  and  6  could  possibly  be  acceptable.  TRLs  7  through  9  would 
appear  to  directly  satisfy  the  intentions  of  the  ACTD  maturity  level  (South,  2003). 


11 


Technology 

Readiness 

Levels 

Description 

Level 

HW/SW 
Necessary  to 
Demon  strate 
Capability 

Environment 

1)  Basic  principles 
observed  anil 
reported 

Lowest  level  of  technology  readiness. 
£cienti  fic  research  begins  to  be 
translated  into  applied  research  and 
development  Examples  might 
include  paper  studies  of  a 
technologv  s.  basic  properties 

Studies 

None 

None 

2)  T  ecbnology 
cuucept  aud  ui 
application 
ful  Uiul.ltell 

Invention  begins  Once  basic 
principles  are  observed,  practical 
applications  -can  be  invented  The 
application  t%  speculative  aud  ihere  is 
no  proof  or  detailed  analysis  to 
support  the  assumption  Examples  are 
still  limited  to  paper  studies 

Studies 

None 

None 

3)  Analytical  and 
experimental 
critical  function 
and  oi 

characteristic 
pa  oof  of  concept 

Active  research  and  development  is 
initiated  Tins  includes  analytical 
studies  and  laboratory  studies  to 
physically  validate  analy  tical 
predictions  of  separate  elements  of 
the  technology  Examples  include 
components  that  are  not  yet  integrated 
or  representative 

Component 

Nonscale  components 
(pieces  of  subsystem) 

Lab 

4)  Component  and 
or  breadboard 

validation  in  lab 
environment. 

Basic  technological  components  are 
integrated  to  establish  that  the  piece-s 
will  work  togethei  This  is  relatively 
low*  fidelity”  compared  to  the 
eventual  system  Examples  include 
integration  of  "ad  hoc"  hardware  in  a 
laboratory. 

Component 

subsystem 

Low  fidelity 
breadboard 

(integration  of  nonscale 
components  not  fully 
functional  or  form  and 

fit) 

Lab 

5)  Component  and  or 
breadboard 
validation  in. 

relevaut 

environment 

Fidelity  of  breadboard  technology 
increases  significantly  The  baisic 
technological  components  are 

Integra  ted  with  rea  vonnbly  rea  listic 
supporting  elements  so  that  the 
technology  can  be  tested  in  a 
simulated  environment  Examples 
include  "high  fidelity'"  laboratory 
integration  of  components. 

Subsystem 

High  fidelity 
bieadboard 
(functionally 
equivalent  but  not  fotm 
and  fit) 

Lab  or  may  include 
demonstration  m 
surrogate  platform 
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Technology 

Readiness 

Levels 

Description 

Level 

HW7SW 
Necessary  to 
Demonstrate 
Capability 

Environment 

6)  System  subsystem 
model  or 
protorvp* 
d  einon  itr  n  ti  on  in 
relevant 
environment 

RefnesoCatree  made]  or  piototvpe 
system.  which  is  well  beyond  die 
bteadboard  tested  tor  technology 
readiness  level  TRL}  5,  is  tested  in  a 
relevant  egustonment.  Represents  a 
major  step  up  in  a  technology's 
demonstrated  readiness  Examples 
include  testing  a  prototype  m  a  high 
fidelity  laboratory  enssroameni  or  in 
a  simulated  operational  environment. 

Sub-system 

Prototype 

(should  be  very  close 
to  form,  fit  and 
function) 

Lab  or  Incited 
dtnaoitttration 

7)  System  prototype 

demonstration  in 
an  operational 
environment 

Prototype  near  or  at  planned 
operational  system.  Represents  a 

major  lies  up  inm  TRL  5.  recusing 
the  demonsaatron  of  an  actual  system 
prototype  in  an  cpeianoiui 
environment,  such  as  in  an  aircraft 
vehicle  a  space  examples  include 
resting  the  prototype  m  2  test  bed 

aaradl 

Sub-system 

Prototype 

(team,  fit  and  function) 

DemoiiOTtioD  m 

rtpi  r^a'Jtivt 
ri"."'finnv'"t  ;udi  X 

tes:b*d 

8)  Actual  system 
completed  and 
(light  "qualified" 
through  test  and 
demonstration 

Technology  hat  been  proven  to  svorh 
m  its  final  form  sad  lurier  expected 
conditions  In  almost  all  cases,  this 

TRL  isxes  e  at;  the  end  «f  true  system 

development.  Examples  sue  aide 
developmental  test  and  evaluanao  of 
the  system  ur  its  intended  weapon 
system  to  de  tessmse  if  ir  meet  s  design 
specif,  canon: 

System 

Field  qualified 
had, vice 

DTJcE  in  acmal 
system  application 

9)  ,V  tual  '.ytltin 
"■flight  proven** 

tfarwgfc  ntmiffal 

ni'/joii  operation? 

Actual  application  of  the  technology 

in  its  final  rVum  -md  nvdpr  mimagi 

ccuditaoo:.  such  as  those  encountered 
or  operational  test  and  evaluation  hr 
almost  all  cases,  this  is  the  end  of  the 
last  “bn*  fixing"  aspects  of  true 
svstem  development.  Examples 
use  lit  de  using  the  system  unde: 
operational  unssma  conations. 

System 

Actual  sytaa  a  final 

form 

Q7&£  ;n  opwatwaal 

ecndiscm 

Table  1.  Technology  Readiness  Level  (From:  Sheldon,  August  2002) 


Since  the  beginning  ACTDs  have  been  scrutinized  heavily  which  has  led  to 
multiple  improvements.  For  example,  transition  manager  were  originally  defined  at  the 
completion  of  the  ACTD  program.  Currently  with  the  more  proactive  execution  attitude 
all  ACTD  programs  must  have  an  established  transition  manager  before  being  considered 
for  execution  approval. 
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The  ACTD  process  evolved  in  1994  in  response  to  the  recommendations  of  the 
Parkard  Commission  of  1986  and  the  Defense  Science  Boards  of  1987,  1990,  and  1991. 
As  can  be  seen  it  has  been  through  many  variations  and  continues  to  change  as  the  global 
environment  changes.  While  these  changes  have  affected  the  process  they  have  not 
effected  its  execution  (South,  2003).  Since  its  inception,  a  total  of  167  ACTDs  have  been 
initiated  from  fiscal  years  1995  through  2007. 

B.  ACTD  PROCESS 

Advanced  Concept  Technology  Demonstrations  (ACTD)  is  a  complicated 
program  that  many  customers  in  the  defense  industry  do  not  understand.  ACTDs  are 
capabilities  demonstration  and  evaluation  programs  in  which  the  development  and 
employment  of  technology  and  innovative  operational  concepts  by  the  military  user  are 
the  primary  focus  (Defense  Acquisition  University,  2006).  The  definition  still  does  not 
explain  much. 

The  ACTD  process  is  a  pre-acquisition  activity  with  a  significant  level  of 
management  oversight,  but  each  program  is  tailored  and  a  much  less  formal  structure 
than  the  actual  acquisition  process  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Advanced  Systems  &  Concepts,  August  2004b).  This  typically  causes  problems  when 
transitioning  from  an  ACTD  to  the  acquisition  process  because  it  is  not  governed  by  laws 
and  regulations  the  same  as  the  nonnal  acquisition  process.  For  this  reason  some  lessons 
learned  from  past  transition  problems  are  addressed  in  the  most  current  version  of  the 
Department  of  Defense  (DoD)  5000.2  document. 

1.  Objective 

An  Advanced  Concept  Technology  Demonstration  (ACTD)  is  a  joint  effort  by  the 
acquisition  and  operational  communities  within  the  DoD.  Typically,  ACTDs  begin  by 
identifying  significant  military  needs,  and  then  matching  them  with  technology  programs 
ready  to  focus  on  a  military  application.  The  emphasis  in  ACTDs  is  a  current  fix  in  order 
to  validate  joint  military  needs.  These  fixes  are  typically  technology  based  and  usually 
include  new  operational  concepts  and  new  organizational  structure.  The  fixes  must  be 
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affordable,  interchangeable,  sustainable,  and  capable  of  being  expanded  as  the 
technologies  and  threats  change  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Advanced  Systems  &  Concepts,  August  2004a).  The  acquisition  approach  over  time  is  an 
important  part  of  the  ACTD  concept. 

The  initial  requirements  and  the  initial  design  reflect  the  performance  achievable 
with  current  technology,  but  provisions  are  included  to  encourage  growth.  The  basic  form 
of  an  ACTD  generally  starts  from  a  collection  of  mature  technologies  or  technology 
demonstration  programs  which  are  key  technologies.  The  technologies  are  combined  and 
integrated  into  a  complete  military  capability.  The  objective  is  to  provide  decision  makers 
an  opportunity  to  fully  understand  the  operational  potential  offered  by  a  proposed  new 
military  capability  before  making  an  acquisition  or  long  term  decision.  This  objective  is 
met  by  developing  useful  prototypes  of  the  proposed  capability  and  providing  those 
prototypes  to  the  war  fighter  for  evaluation  (Office  of  the  Deputy  Under  Secretary  of 
Defense  for  Advanced  Systems  &  Concepts,  August  2004a).  The  war  fighter  first 
develops  operational  concepts  designed  to  fully  challenge  the  proposed  capability,  and 
then  uses  the  prototypes  and  associated  operational  concepts  in  realistic  military  exercises 
to  assess  the  resulting  military  utility.  During  the  assessment  of  the  ACTD,  the  user  also 
determines  the  broad  statement  of  need.  This  was  first  introduced  in  the  proposal  of  the 
ACTD,  and  should  have  a  definite  set  of  operational  requirements  that  can  support  a 
follow-on  acquisition  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced 
Systems  &  Concepts,  August  2004a).  At  the  completion  of  an  ACTD,  the  remaining 
systems  used  in  the  evaluation  process  are  left  with  the  user  to  provide  a  temporary 
capability  or  in  some  cases  to  fulfill  the  total  current  need. 

Each  ACTD  should  provide  a  serious  need  for  new  or  increased  military 
capability.  This  need  is  usually  provided  by  the  operational  war  fighting  community 
(JCS,  CINCs,  Service  operational  organizations).  Although  some  ACTDs  focus  on  a 
service  specific  capability,  the  highest  priority  in  the  selection  process  is  placed  on  joint 
capabilities.  ACTDs  have  become  an  important  vehicle  for  addressing  joint  needs  (Office 
of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August 
2004a). 
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A  primary  ground  rule  for  any  ACTD  is  the  active  participation  of  a  sponsor  or 
user  organization,  in  partnership  with  a  service  which  will  serve  as  the  Technical 
Management  Office.  A  proposal  for  an  ACTD  must  identify  and  develop  this  user  and 
developer  partnership  before  consideration  can  be  given.  The  initiation  is  either  by  the 
acquisition  community,  or  by  the  war  lighter  community.  The  interests  of  the  war  lighter 
are  very  important  and  the  guidelines  regarding  ACTDs  are  considered  flexible. 

The  Deputy  Under  Secretary  of  Defense  (Advanced  Technology)  (DUSD/ 
AS&C)  has  the  oversight  responsibility  for  the  ACTD  program.  He  is  responsible  for 
developing  and  issuing  guidance  regarding  the  ACTD  program,  for  evaluating  candidates 
and  approving  new  ACTDs.  He  is  also  going  to  provide  oversight,  support  and 
evaluation  of  ongoing  ACTDs  (Defense  Acquisition  University,  2006.  This  document 
describes  the  process  for  formulating,  and  evaluating  ACTD  candidates,  and  for 
approving  and  initiating  ACTDs. 

The  goals  of  the  ACTD  processes  are  to  accelerate  and  facilitate  responses  to 
priority  military  needs  with  a  combination  of  new  and  fielded  hardware  and/or  software, 
confirming  that  transformational  technology  is  appropriate  for  military  use,  develop 
CONOPS  trough  resources,  and  creating  an  organizational  structure  that  satisfies  those 
needs  (South,  2003). 

To  satisfy  the  objectives  guidelines  have  been  developed  which  apply  to  both  the 
proposed  capability  and  to  the  program  for  developing  and  evaluating  the  capability. 
These  guidelines  have  been  referred  to  as  the  ACTD  selection  criteria,  they  are  intended 
to  provide  guidance  for  formulation  of  candidates,  as  well  as  structure  during  the  ACTD 
process.  The  criteria  are  as  follows:  (Office  of  the  Deputy  Under  Secretary  of  Defense 
for  Advanced  Systems  &  Concepts,  August  2004a). 

1.  The  timeframe  for  completing  the  evaluation  of  military  utility  is  2-4 
years. 

2.  The  technology  should  be  sufficiently  mature. 

3.  Provide  a  potentially  effective  response  to  a  priority  military  need. 

4.  Lead  service/agency  has  been  designated. 

5.  Risks  have  been  identified. 
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6.  Demonstrations  or  exercises  have  been  identified. 

7.  Funding  is  sufficient. 

8.  Developer  is  ready  to  prepare  a  plan  that  covers  all  essential  aspects. 

There  are  other  factors  that  should  be  considered  during  the  ACTD  formulation 
phase.  Although  these  are  not  selection  criteria  per  se,  they  do  lead  to  better  alignment 
with  the  objectives  of  the  ACTD  program  and  may  affect  the  level  of  support  a  given 
candidate  receives  during  the  selection  process.  As  indicated  earlier,  the  emphasis  in 
ACTDs  is  on  near-term  responses  to  the  need,  responses  that  are  affordable, 
interoperable,  sustainable,  and  capable  of  being  evolved  as  the  technology  and  threat 
change  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  & 
Concepts,  August  2004a). 

2.  Selection  Process 

The  ACTD  process  is  marked  by  three  basic  phases:  selection  of  the  projects, 
demonstration  of  the  technologies,  and  residual  use  of  prototypes  or  the  transition  to 
acquisition  programs  (United  States  General  Accounting  Office,  2002).  The  selection 
process  begins  with  the  submission  of  proposed  Advanced  Capability  Technology 
Demonstration  in  response  to  the  data  call  issued  by  the  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology)  (DUSD(AS&C)  in  October  of  each  year.  When 
submitting  a  particular  technology/concept  as  an  ACTD,  a  one  to  three  page  description 
of  the  proposal  should  be  provided  and  should  include  the  following  infonnation: 

Describe  the  perceived  military  need,  urgency  of  timing,  and  potential  utility  of 
the  candidate  system. 

Paragraph(s)  describing  the  basic  technology/concept. 

Paragraph  describing  the  type  of  demonstration  envisioned. 

Participants  in  the  ACTD.  To  what  degree  will  the  proposed  ACTD  support 
joint/combined  operations? 

Overall  funding  required,  proposed  funding  sources  and  the  schedule  for  the 

ACTD. 
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Paragraph  describing  perceived  technical,  funding,  and  schedule  risks  of  the 
proposal. 

Is  the  ACTD  primarily  directed  towards  Dominant  Maneuver,  Precision 
Engagement,  Full-Dimensional  Protection,  or  Focused  Fogistics? 

Proposed  Fead  Service/Agency  and  User-Sponsor. 

Briefly  describe  envisioned  residual  assets  and  ACTD  transition  strategy. 

Point(s)  of  contact  for  the  ACTD  submission.  Include  name,  rank,  organization, 
phone  number,  fax  number,  and  addresses  (mail  /  e-mail).  (Office  of  the  Deputy  Under 
Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004a). 

Once  a  developer  and  user  team  has  submitted  the  information  described  above, 
the  candidate  review  process  begins.  A  specific  individual  within  the  ODUSD(AS&C)  is 
designated  to  serve  as  the  point-of-contact  (POC)  for  each  ACTD  candidate.  That  person 
will  work  with  the  individual  organization  to  ensure  the  proposal  is  complete  and 
coordinate  the  review  process  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Advanced  Systems  &  Concepts,  August  2004a).  A  typical  schedule  for  this  process  is 
shown  in  the  Figure  1  below. 

The  first  step  in  the  process  is  to  prepare  a  briefing  package  (no  longer  than  30 
minutes)  to  be  presented  first  to  the  DUSD(AS&C)  due  the  January  after  the  data  call  is 
pronounced.  The  briefing  should  define  the  top  level  mission  need,  the  ACTD  objectives, 
operational  concept,  technical  approach,  time  scale,  anticipated  program  cost  (with 
funding  sources  identified  and/or  required,  including  out-year,  OSD  ACTD  funds),  the 
management  structure,  the  primary  participants,  the  anticipated  capability  and,  should 
address  the  "ACTD  Selection  Criteria.”  (Office  of  the  Deputy  Under  Secretary  of 
Defense  for  Advanced  Systems  &  Concepts,  August  2004a).  Doing  this  first  step 
correctly  is  the  key  to  success.  If  this  step  is  not  complete  then  the  whole  proposal  is  at 
risk.  The  first  impression  is  what  makes  or  breaks  the  success  of  the  proposal. 
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The  briefing  should  be  presented  by  both  the  user  and  developer  teammates.  The 
user  outlines  the  mission  need  and  operational  concept,  and  the  developer  presents  the 
technical  concept  and  programmatic  approach,  highlighting  a  clear  statement  of  the  end 
product  (the  interim  capability).  The  USD(AS&C)  staff  stands  ready  to  assist  in  the 
development  of  this  briefing,  as  well  as  the  scheduling,  as  necessary  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004a). 
With  the  USD  staffs  assistance  this  initial  submission  should  be  perfect. 

The  result  of  this  initial  briefing  will  be  one  of  the  following  courses  of  action:  (a) 
acceptance  of  the  ACTD  as  a  formal  "candidate"  and  scheduling  for  presentation  to  the 
full  AT/BC  principles  for  discussion  and  recommendation;  (b)  critique  by  DUSD(AS&C) 
and  request  for  revision  and  follow-up  presentation;  or  (c)  detennination  that  the  concept, 
for  whatever  reason,  is  outside  the  scope  of  the  ACTD  process  (Office  of  the  Deputy 
Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004a). 

After  the  DUSD(AS&C)  briefing  the  proposal  goes  through  a  couple  initial 
reviews  (Advanced  Technology  Breakfast  Club  (AT/BC)  initial  review  from  February  to 
March,  then  a  status  review  with  the  hill  in  April)  before  being  submitted  to  the  Breakfast 
Club  (BC)  or  a  detailed  review.  The  BC  will  meet  at  the  beginning  of  April  and  usually 
finish  up  by  the  beginning  of  June.  BC  is  comprised  of  the  senior  Science  &  Technology 
(S&T)  representatives  from  the  services,  Director  of  Defense  Research  &  Engineering 
(DDR&E),  Command,  Control,  Communication  &  Intelligence  (C3I),  and  selected 
Department  of  Defense  (DoD)  agencies,  representatives  from  the  Operational 
Departments  of  the  Services  and  from  the  Joint  Staff  (Defense  Acquisition  University, 
2006).  After  the  BC  is  done  with  the  proposal  it  is  then  submitted  to  the  Joint  Staff  for 
another  review  that  takes  place  from  June  to  August. 

The  DUSD(AS&C)  will  consider  the  recommendations  of  the  AT/BC  and  the 
Joint  Staff  and  make  the  decision  to  retain  the  specific  ACTD  candidate  for  presentation 
to  the  Joint  Requirement  Oversight  Counsel  (JROC).  The  JROC  reviews  and 
recommends  prioritization  of  ACTD  candidates  based  on  military  need.  At  this  point, 
information  on  the  candidates  is  provided  to  the  Congressional  Authorization  and 

Appropriations  Committees  to  support  their  committee  marks.  A  final  review,  termed  the 
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'Final  Scrub',  is  then  conducted  prior  to  the  start  of  the  fiscal  year  (Defense  Acquisition 
University,  2006).  Of  the  candidates  selected  by  the  Joint  Staff  and  OSD  are  most 
deserving  of  ACTD  status.  The  focus  of  this  review  is  once  again  on  the  selection 
criteria,  with  the  addition  of  two  other  topics;  transition  strategy  and  proposed  ACTD. 

The  ACTD  list  is  then  coordinated  with  the  Vice  Chairman,  Joint  Chief  of  Staff 
(JCS)  and  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  before  the  final 
ACTD  Implementation  Directives  (ID)  for  the  approved  ACTDs  are  signed  by  the 
DUSD(AS&C)  (Aldridge,  April  12,  2002). 


ACTD  FY-2007  ACTD/JCTD  Program  Timeline 


Selection  Process  for  the  FY  2007  ACTD/JCTD  Candidates 


ACTD 

Proposals 

AT&BC 

Status 

Due  to  DUSD 

Initial 

Review  with 

Review 

Hill 

Jdl  1 

Feb/Mar 

Apr 

USD(AT)  &  VC 
JCS  Approval 
Oct 


DUSD(AT) 

Announces 

Selections 

Oct(late) 


Definitions: 

•  FCB:  Functional  Capability  Board 

•  JROC:  Joint  Requirements  Oversight 
Council 

•  CoCOM:  Combatant  Commander 


ACTD  to  JCTD  conversion  occurs  after  JROC  validation 


Figure  1.  Program  Timeline  (From:  Carson  PPT,  2006) 


At  the  same  time  the  briefing  is  presented  to  the  BC,  a  draft  Implementation 
Directive  should  be  in  work  and  presented  to  AS&C  staff  once  the  candidate  is  selected. 
The  final  selection  of  ACTDs  will  not  occur  until  the  Defense  Appropriations  Bill  has 
been  signed.  However,  soon  after  that  time  the  final  ACTD  approvals  can  be  granted. 
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Approvals  are  in  the  form  of  the  DUSD/AS&C  signature  on  the  Implementation 
Directives,  so  it  is  imperative  the  staffing  of  the  Implementation  Directives  be  completed 
in  a  timely  fashion  (Defense  Acquisition  University,  2006).  Even  though  there  is  not  a 
set  timeline  on  the  ID,  it  is  the  highest  priority  once  the  selections  occur.  A  late 
document  will  risk  no  final  signature  and  a  cancelled  ACTD. 

The  final  step  of  ACTD  process  is  the  completion  of  the  ACTD  Management  Plan 
(MP).  The  items  that  are  addressed  in  the  management  plan  include  the  following: 
(Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004a). 

1 .  The  objectives  that  the  ACTD  must  demonstrate. 

2.  The  overall  approach. 

3.  The  concept  and  technical  approach  of  the  ACTD 

4.  The  programmatic  and  organizational  approach,  which  includes  the  key 
decision  makers. 

5.  The  approval  agencies 

6.  The  endorsements  of  the  ACTD  participants. 

7.  Any  modifications  associated  with  the  ACTD 

The  process  of  working  out  the  details  of  the  Management  Plan  to  the  satisfaction 
of  all  involved  will  take  some  time  and  this  process  is  viewed  as  a  very  productive 
element  of  the  ACTD  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced 
Systems  &  Concepts,  August  2004a).  It  is  necessary  for  the  Plan  to  receive  full 
endorsement  within  90  days  or  less  after  approval  of  the  Implementation  Directive. 

For  this  project,  the  most  important  section  of  the  management  plan  is  the 
programmatic  and  organizational  approach,  item  4,  which  include  key  players  and  the 
transition  plan. 

The  initial  meeting  of  the  ACTD  Oversight  Group  should  be  scheduled  during  the 
first  year  of  the  program  to  confirm  the  planned  direction  of  the  program  and  if  necessary 
to  resolve  any  outstanding  issues  relating  to  the  Management  Plan  (Office  of  the  Deputy 
Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004a).  The 
Management  Plan  is  intended  to  provide  a  baseline  program  definition,  as  well  as,  a 
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practical  and  flexible  learning  environment  in  which  operations  and  technical  concepts 
can  be  traded  off  and  refined  prior  to  entering  the  formal  acquisition  process.  Only 
significant  modifications  to  the  Plan  need  be  approved  by  the  ACTD’s  Oversight  Group 
(Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004a). 

3.  ACTD  vs.  Acquisition  Funding 

ACTD  program  managers  must  obtain  ACTD  and  any  follow-on  acquisition 
funding  through  the  Planning  Program  and  Budgeting  System  (PPBS)  just  like  traditional 
acquisition  program  managers.  While  traditional  acquisition  programs  should  be  fully 
funded  in  the  Future  Years  Defense  Plan  (FYDP),  ACTD  programs  are  not  required  to 
include  funding  for  post-ACTD  activity  in  the  FYDP.  At  first  glance  this  benefit  of  not 
funding  additional  research  and  development  (R&D)  or  any  production  effort  may  appeal 
to  the  Services  and  OSD  in  a  fiscally  constrained  environment;  however,  it  is  not 
practical  and  creates  problems  as  ACTDs  transition  to  acquisition  programs  (Mol,  1998). 

In  reality,  post-ACTD  financial  planning  must  be  accomplished  during  the  ACTD 
since  the  acquisition  Milestone  Decision  Authority  (MDA)  will  only  transition  the 
program  from  an  ACTD  to  an  acquisition  program  if  the  follow-on  effort  is  fully  funded. 
This  problem  is  recognized  within  the  acquisition  community  since  it  affects  not  only  the 
ACTD  and  its  follow-on  acquisition  effort,  but  also  other  modernization  programs 
competing  for  the  same  scarce  funding  (Mol,  1998).  As  mentioned  earlier  if  the  initial 
proposal  is  not  complete  on  time  the  ACTD  could  risk  failure  due  to  a  lack  of  funds. 
This  is  why  that  first  step  in  the  selection  process  is  so  critical  to  the  existence  of  ACTDs. 

OSDs  ACTD  guidelines  offer  three  strategies  to  deal  with  this  problem  if  the 
funding  was  not  properly  planned  for.  First,  the  services  can  appeal  directly  to  OSDs 
Defense  Resources  Board  (DRB)  to  include  funding  for  the  follow-on  acquisition  effort. 
If  this  brute  force  method  is  successful,  it  means  OSD  will  transfer  funding  from  an 
approved  program  to  the  new  ACTD  follow-on  effort.  This  method  disrupts  the  PPBS 
process  by  placing  new  funding  requirements  very  late  in  the  process  after  priority  and 
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funding  issues  should  have  previously  been  resolved  within  the  services  (Mol,  1998). 
This  method  is  not  very  popular  and  most  likely  will  not  happen  unless  it  comes  from 
very  high  in  the  chain-of-command. 

The  second  alternative  suggests  the  acquisition  strategy  contain  a  two-year  gap 
between  the  completed  ACTD  and  the  beginning  of  the  fonnal  acquisition  process.  This 
gap  allows  program  managers  time  to  obtain  funding  through  the  normal  two-year  POM 
process.  While  this  suggestion  creates  efficiency  within  the  PPBS  process,  it  is  likely  to 
break  the  program  and  cause  its  cancellation  due  to  the  increased  contractor  shut  down 
and  startup  costs  (Mol,  1998).  Historically  this  has  been  a  major  cause  of  ACTDs  to  fail. 
As  we  will  see  later  there  has  been  a  patch  to  fix  this  problem  by  using  OSD  funds  from 
the  RTD&E  funds  budgeted  for  this  purpose.  With  that  being  said  it  still  needs  to  be 
planned  for  in  the  initial  proposal. 

The  third,  and  probably  most  attractive  solution  offered,  is  to  assume  success.  If 
the  acquisition  strategy  includes  this  course  of  action,  the  services  must  insert  an 
acquisition  cost  estimate  into  the  PPBS  process  before  the  ACTD  testing  is  complete  and 
before  the  user  has  had  an  opportunity  to  make  an  operational  assessment  (Mol,  1998). 
Unfortunately,  not  having  the  test  results  will  build  uncertainty  into  the  cost  estimate  and 
increase  the  funding  since  results  obtained  in  the  last  year  or  two  of  the  ACTD  are  the 
most  important.  During  this  critical  time  DOD  will  determine  the  production 
configuration,  the  type  of  funding  required  (R&D  vs.  Production),  and  the  scope  of  any 
future  effort  (Mol,  1998).  Consequently,  the  Services  may  be  reluctant  to  fund  any 
follow-on  effort  given  the  ACTD’s  unpredictable  future. 

A  specific  example  would  be  programming  funds  in  the  POM  cycle  for  follow-on 
production  of  an  ACTD  where  success  is  anticipated,  such  as  for  Global  Hawk,  even 
though  flight  testing  has  not  yet  demonstrated  high  military  utility  (Mol,  1998).  The 
Army  already  has  a  similar  strategy  in  place  to  fund  emerging  technologies,  such  as 
Advanced  Technology  Demonstrations  (ATDs)  and  Advanced  War  fighting  Experiments 
(AWEs).  During  the  development  process  of  the  FY98-03  POM,  the  Army  established  a 
Task  Force  21  budget  line,  with  RDT&E  funds  identified  and  submitted  in  the  FY98 

budget  request.  The  establishment  of  the  RDT&E  line,  to  support  Force  21  requirements, 
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provides  the  service  the  flexibility  to  leverage,  exploit  and  transition  new  technologies, 
buy  prototype  systems,  and  put  them  in  the  hands  of  the  soldiers  quickly  (Mol,  1998). 

In  the  end,  the  funding  rules  are  different  between  the  two  systems.  PPBS  reality 
dictates  ACTD  programs  and  it  must  have  a  plan  and  program  for  follow-on  acquisition 
funding  in  the  FYDP  to  maintain  program  stability.  ACTD  program  managers  and  the 
service  headquarters  must  incorporate  their  budgets  into  the  PPBS — just  like  traditional 
acquisition  programs. 

4.  Current  Funding  Process 

Programmatic  flexibility  and  speed  in  adjusting  to  change  are  critically  important 
to  success  with  a  program  as  technologically  intensive  as  the  ACTD.  In  the  current 
environment,  technology  is  accelerating  at  a  tremendous  rate.  Our  speed  and  flexibility  to 
leverage,  exploit,  and  transition  mature  or  emerging  technologies  into  the  operational 
force  structure  is  hampered  by  resource  and  budget  constraints  (e.g.,  the  inability  to 
perform  timely  programming  of  funding  during  the  Program  Objective  Memorandum 
(POM)  process)  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems 
&  Concepts,  August  2004b).  If  the  selection  process  is  used  correctly  it  can  minimize 
some  of  these  resource  and  budgeting  issues  that  become  major  road  blocks  for  future 
requirements. 

At  the  time  a  proposed  ACTD  is  approved,  the  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology)  also  approves  the  funding  for  an  ACTD,  to  include  any 
supplemental  funding  provided  by  OSD.  The  Executing  Agent  will  designate  an  ACTD 
Technical  Manager  (TM)  who  is  responsible  for  managing  the  execution  of  all  funds 
associated  with  an  ACTD  (Mol,  1998).  It  is  also  the  responsibility  of  the  TM  to  develop  a 
life  cycle  cost  estimate  for  the  system  to  serve  as  a  basis  for  planning,  programming,  and 
budgeting  of  the  resources  by  the  Lead  Service  for  subsequent  acquisition. 

Funding  for  ACTDs  can  currently  be  planned,  programmed,  and  budgeted 

through  two  sources:  1)  The  Military  Departments  or  Agencies  supplying  the  underlying 

technologies  provide  the  funding  associated  with  those  technology  programs,  and  2)  OSD 

can  supplement  the  service  or  agency  funding  to  cover  cost  in  three  areas:  a)  added  costs 
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incurred  when  other  technology  program  funds  are  redirected  to  support  the  ACTD;  b) 
costs  due  to  any  requirement  to  provide  additional  quantities  of  hardware;  and  c)  cost  for 
technical  support  for  two  years  of  field  operations  following  the  ACTD  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004b). 
Although  OSD  provides  start  up  funding  for  ACTDs,  the  military  services  and  defense 
agencies  are  ultimately  responsible  for  financing  the  acquisition  and  support  of 
equipment  for  the  ACTD  (United  States  General  Accounting  Office,  2002).  However, 
funding  to  support  the  follow-on  activity  (development,  full  rate  production,  or  purchase 
of  additional  quantities  of  commercial  items)  is  not  typically  funded  in  OSD  or  the 
Service/Agency  until  the  ACTD  demonstrates  the  military  utility  of  the  capability  being 
assessed  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  & 
Concepts,  August  2004b).  This  lack  of  prior  funding  creates  a  significant  challenge  that 
must  be  addressed  as  part  of  the  transition  effort. 

Proper  funding  of  ACTDs  is  a  critical  component  for  successful  program 
execution.  Funds  for  ACTDs  are  submitted  in  the  RDT&E  Presidential  Budget  Request. 
A  majority  of  these  resources  are  pulled  from  the  BA-3  Advanced  Technology 
Development  funds.  Some  of  the  budgeting  requirements  include  funding  that  must  be 
sufficient  to  complete  the  planned  assessment  of  utility.  Budget  request  must  be 
developed  and  submitted  as  a  part  of  the  proposed  ACTD.  The  budget  must  identify  all 
costs  associated  with  the  design  and  development  of  the  prototype  system,  all  additional 
units  required  in  the  ACTD,  all  exercises  that  must  be  paid  by  the  project,  and  test 
support  costs  including  any  modeling  simulation  and  analysis  needed  to  support  the 
utility  assessment  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced 
Systems  &  Concepts,  August  2004a).  The  budget  must  also  include  costs  related  to 
planning  and  preparations  for  transition  into  acquisition,  as  well  as  the  cost  to  provide 
technical  support  for  the  first  two  years  of  fielding  the  residuals.  The  lead  service  is 
assumed  to  budget  for  all  support  costs  beyond  that  point  (Defense  Acquisition 
University,  2006). 
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Both  the  Implementation  Directive  (ID)  and  Management  Plan  (MP)  require  a 
“break-out”  of  resources.  Historically,  OSD  has  provided  15-20%  of  the  total  funding 
while  the  services  involved  provide  the  rest.  It  is  the  goal  of  AS&C  to  provide 
approximately  20%  of  the  total  funding  and  no  more  than  30%  of  the  actual  cash  funding 
for  any  particular  ACTD  (not  including  coalition  partners)  (Peterson,  2005).  In  addition, 
properly  spreading  the  resources  is  a  critical  element  of  the  program.  AS&C  resources 
should  be  spread  across  the  life  of  the  project,  with  no  more  than  half  of  its  funding 
requested  in  the  first  two  years.  At  the  discretion  of  the  DUSD  (AS&C)  these  metrics 
may  be  waived,  but  it  should  be  the  exception  and  considered  only  on  a  case  by  case 
basis  (Peterson,  2005).  Proposals  for  OSD  funding  should  be  coordinated  with 
ODUSD(AS&C)  during  the  fonnulation  phase.  Figure  2  shows  the  current  ACTD 
funding  model  in  place  today  and  highlights  the  challenges  involved  in  the  process. 


Current  ACTD  Funding  Model 


Challenges: 

•  Front-end  (start-up)  and  tail-end 
(transition)  funding  issues  create  serious 
PPBE  challenges. 

•  Little  incentive  for  Service  participation  as 
New  ACTDs  create  immediate  unfundeds. 

•  Significant  start-up  and  demonstration 
delays  after  JROC  decision:  Average  6 
month  delay  waiting  for  Implementation 
Agreements. 

•  Many  different  Program  Elements  fund 
ACTDs  (Little  visibility  at  Service  level — 
accountability  challenges) 

•  Projects  require  sustained  commitment 
of  resources  once  initiated. 

•  Unfunded  Requirements  (UFRs)  during 
execution  cause  significant  risk  and 
disruption  as  OSD  tries  to  “share”  the 
UFRs  with  stakeholders. 

•  Even  successful  demonstrations  risk 
waiting  2  years  (or  more)  for  resources  to 
be  programmed  via  rigid  PPBE  process. 


Goal  is  to  initiate  ACTDs  within  months  of  a  JROC  approval.  However,  two 
year  PPBE  process  creates  Service  challenge  in  funding  new  ACTDs. 
(Difficult  to  achieve  before  the  third  year) 


OSD  AS&C  Cash  Resources  (-30%) 


XX 


All  other  Sources  (-70%) 


USAF 


PExxxx 

PExxxx 

PExxxx 

PExxxx 


USMC 


PExxxx 

PExxxx 

PExxxx 

PExxxx 


Agency 


PExxxx 

PExxxx 

PExxxx 

PExxxx 


Figure  2.  Current  Funding  (From:  Carson  PPT,  2006) 


26 


5. 


Conclusion 


Although  this  system  has  its  problems  in  the  end  it  has  been  a  relatively 
successful  program  for  the  DoD.  They  claim  roughly  71%  of  the  ACTDs  transition  at 
least  one  product  to  a  program  of  record  or  directly  to  the  war  fighter.  DoD  would  like  to 
see  that  number  climb  to  over  80%  (TechLink,  2005).  There  has  been  a  considerable 
amount  of  effort  put  into  shrinking  the  time  it  takes  to  get  a  product  through  the 
acquisition  process.  This  program  was  a  way  to  get  high  priority  items  in  the  hands  of 
the  war  fighter  on  limited  resources.  After  looking  at  the  process  the  main  problem  is  the 
budget  because  it  is  not  in  line  with  the  typical  PPBE  process. 

Figure  3  shows  how  serious  Congress  has  started  to  take  this  program.  Prior  to 
2001  the  appropriations  were  much  lower  than  what  was  requested.  Since  then,  Congress 
has  issued  appropriations  above  what  was  requested.  This  is  a  good  sign  for  future 
programs  brought  into  the  ACTD  process.  Improvements  will  continue  to  be  made  as 
time  goes  on  with  the  emphasis  on  transitioning  the  ACTDs  into  the  acquisition  process. 


ACTD  Historical  Funding  Profile 
Budget  Vs.  Appropriation 


Starting  in  FY  2001  Congress  fully  supports  ACTD  Budget  Request 


Figure  3.  Historical  Funding  (From:  Carson  PPT,  2006) 
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c. 


REVIEW  OF  PROGRAMS 


ACTDs  represent  a  bold  departure  from  the  traditional  research  and  development 
acquisition  cycle  that  can  take  15  years  (Kaminski,  March  1997)  to  field  a  new  weapon 
system.  ACTDs  typically  have  a  two  to  four  year  life  span  as  standalone  demonstration 
activities  (South  thesis).  Since  the  beginning  of  the  ACTD  process  there  have  been  167 
programs  approved  through  FY07.  Of  these,  93  have  successfully  completed  the 
demonstration  phases  and  55  are  still  in  process.  Of  those  167  programs  16  have  been 
tenninated  due  to  lack  of  military  utility  or  immaturity  while  three  have  been  place  on 
hold.  Seventy-four  have  been  placed  in  the  "transitioned  on  record"  category  indicating  a 
successful  transition.  ACTDs  initiated  to  date  are  presented  in  Table  2  (Carson,  2007). 
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FY  95  ACTD 

FY96  ACTD 

FY97ACTD 

FY  98  ACTD 

Advanced  Joint  Planning 

Battle  Field  Awareness 
and  Data  Dissemination 

Integrated  Collection 
Management 

Migration  Defense 
Intelligence  Threat  Data 
System 

Precision  SIGINT 

Targeting  System 

Semi-Automated  IMINT 
Processing 

Information  Operations 
Planning  Tool 

Joint  Continuous  Strike 
Environment 

Synthetic  Theater  of  War 

Joint  Logistics 

Consequence 

Management 

Adaptive  Course  of 
Action 

Low-Life-Cycle-Cost 
Medium-Lift  Helicopter 

Counter  Sniper 

Joint  Advanced  Health 
and  Usage  Monitoring 
System 

C4I  for  Coalition 
Warfare 

Kinetic  Energy  Boost- 
Phase  Intercept 

Miniature  Air-Launched 
Decoy 

Rapid  Terrain 

Visualization 

Space-Based  Space 
Surveillance  Operations 

Medium-Altitude 

Endurance  UAV 
(Predator) 

Combat  Vehicle 
Survivability 

Chemical  Add-On  to  Air 
Base/Port  Biological 
Detection 

Information  Assurance: 
Automated  Intrusion 
Detection  Evnironment 

High- Altitude  Endurance 
UAV 

Navigation  Warfare 

Military  Operations  in 
Urban  Terrain 

Theater  Precision  Strike 
Operations 

Cruise  Missile  Defense 
Phase  I 

Tactical  High-Energy 

Laser 

Extending  the  Littoral 
Battlespace 

Unattended  Ground 
Sensors 

Precision/Rapid  Counter- 
MRL 

Tactical  UAV 

Counterproliferation  II 

Precision  Targeting 
Identification 

Joint  Countermine 

Air  Base/Port  Bilogical 
Detection 

Joint  Modular  Lighter 
System 

Rapid  Force  Prjection 
Initiative 

Combat  Identification 

Line-of-Sight  Anti-Tank 

Counterproliferation  I 

Joint  Biological  Remote 
Early  Warning  System 

Link- 16 

Total  11 

Total  12 

Total  9 

Total  13 
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FY99  ACTD 

FY00  ACTD 

FY01  ACTD 

FY02  ACTD 

Battle  Damage 

Assessment  in  the  Joint 
Targeting  Toolbox 

CINC  21 

Active  Network  Intrusion 
Defense 

Active  Denial  System 

Coherent  Analytical 
Computing  Environment 

Coalition  Aerial 
Surveillance  & 
Reconnaissance 

Adaptive  Batlespace 
Awareness 

Advanced  Notice 

Common  spectral 

MASINT  Exploitation 

Comm/Nav  Outage 

Forecast  System 

Advanced  Tactical  Laser 

Agile  Transportation 

Compact  Environmental 
Anomaly  Sensor  II 

Computerized 

Operational  MASINT 
Weather 

Advanced  Technology 
Ordnance  Surveillance 

Coalition  Information 
Assurance  Common 
Operational  Picture 

Force  Medical 
Protection/Dosimeter 

Content-Based  Info 

Security 

Area  Cruise  Missile 

Defense 

Contamination 

Avoidance  at  Seaports  of 
Debarkation 

Human  Intelligence 

Support  Tools 

Global  Monitoring  of 

Space  ISR  Systems 

Coalition  Combat  ID 

Expendable  Unmanned 
Aerial  Vehicle 

Joint  Medical 
Operations/Telemedicine 

Ground-to-Air  Passive 
Survillance 

Coalition  Theater 

Logistics 

Homeland  Security 
Command  and  Control 

Joint  Thheater  Logistics 

Joint  Intelligence, 
Surveillance  & 
Reconnaissance 

Coastal  Area  Protection 
System 

Hyperspectral  Collection 
and  Analysis 

Personnel  Recovery 

Mission  Software 

Multiple  Link  Antenna 
System 

Hunter  Standoff  Killer 

Team 

Joint  Explosive  Ordnance 
Disposal 

Small  Unit  Logistics 

Quick  Bolt 

Joint  Area  Clearance 

Language  and  Speech 
Exploitation  Resource 

Theater  Air  &  Missile 
Defense  Interoperability 

Restoration  of  Operations 

Loitering  Electronic 

Warfare  Killer 

Micro  Air  Vehicle 

Tri-Band  Antenna  Signal 
Combiner 

Network-Centric 
Collaborative  Targeting 
(formally  NCCIS&R) 

Pathfinder 

Personnel  Recovery 
Extraction  Survivability 
aided  by  Smart  Sensors 

SIGINT  Processing 

Tactical  Missile  System 
Penetrator 

Space-Based  MTI 

Theater  Integrated 

Planning  System 

Thermobarics 

Total  11 

Total  12 

Total  15 

Total  15 
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FY03  ACTD 

FY04  ACTD 

FY05  ACTD 

FY06  ACTD 

Adaptive  Joint  C4ISR 

Node 

Agile  Rapid  Global 

Combat  Support 

Actionable  Situational 
Awareness  Pull 

Counterintelligence 
Architecture 
Modernization  Progam- 
Intell  Ops 

Counter  Bomb/Counter 
Bomber 

Advanced  Tactical 

Targeting  Technology 

Operations  System 

Management  Planning 
Tool 

Deployable  Cargo 

Screening 

Coalition  Reception 

Staging  &  Onward 
Movement 

Chemical  Unmanned 
Ground  Reconnaissance 

Awareness 

Foliage  Pen  Syn  App 

Rad 

Environment 

Epidemic  Outbreak 
Surveillance 

Event  Management 
Framework 

Gridlock 

Future  Tactical  Truck 
System 

Engagement 

Architecture 

High  Altitude  Airship 

Joint  Precision  Airdrop 
System 

Joint  Enhanced 

Explosion  Resistant 

Coating  Exploitation 

Focused  Lethality 
Munition 

Joint  Blue  Force 

Situational  Awareness 

Control 

Joint  Force  Projection 

Joint  Enable  Theater 
Access-Sea  Ports  of 
Debarkation 

Midnight  Stand 

MAGNUM 

Theater 

Systems 

Night  Vision  Cave  & 

Urban  Assault 

Joint  ISR  Interoperability 
Coalition 

Rapid  Airborne 

Reporting  &  Exploitation 

Large  Data 

Overwatch 

Man  Portable  Threat 
Warning  System 

Sea  Eagle 

Counter  Obscured 
Targets 

Tactical  IFSAR  Mapping 

Protected  Landing  and 
Take-off 

Sea  TALON 

Node  Management  and 
Deployable  Depot 

Theater  Support  Vessel 

Global  Reach 

Demonstrator 

Capability 

Tunnel  Target  Defeat 

Theater  Effects  Based 
Operations 

TACSAT-2  Roadrunner 

Small  Unmanned  Aerial 
System 

Urban  Recon 

Gunship  standoff 

Precision  Munition 

Weapons  Data  Link 
Network 

Total  14 

Total  13 

Total  15 

Total  13 
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FY07  ACTD 


Airborne  Weapons 
Surveillance  System 

Coalition  Mobility 
System 

Global  Observer 

IP  Routers  in  Space 

Joint  Multi-Mission 
Electro-Optical  System 

Joint  Surface  Warfare 

Mapping  the  Human 
Terrain 

Maritime  Auto  Super 
Track  Enhance  Reporting 

Smart  Threads  Integrated 
Radiation  Sensors 

Tactical  Service 
Providers 


Total  10 

Table  2.  ACTD  Initiated  by  Fiscal  Year  (From:  DoD  Release  95-07) 
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D.  CHAPTER  SUMMARY 

This  chapter  began  with  a  review  of  the  ACTD  process  initiated  in  1995.  The 
background  information  provided  the  history  on  how  the  process  got  started.  With  that 
information  we  were  able  to  provide  detailed  procedures  on  how  an  ACTD  gets  selected, 
funded,  and  eventually  transitioned  into  the  acquisition  process. 

The  most  significant  items  associated  with  the  establishment  and  methods  of  a 
given  ACTD  program  include: 

1.  ACTDs  were  initiated  to  reduce  the  acquisition  cycle  time  and  speed  the 
delivery  of  advanced  capabilities  to  the  war  fighter. 

2.  The  process  utilized  to  establish  and  execute  ACTDs  has  continually 
evolved  since  1995  to  increase  the  potential  for  success. 

3.  Technology  maturity  above  readiness  level  5,  as  defined  in  (Table  1),  is 
critical  to  the  expectation  and  ACTD  will  successfully  transition  into 
acquisition. 

4.  Three  categories  of  ACTDs  exist:  software  systems,  weapon  and  sensor 
systems,  and  system-of-systems. 

5.  Exit  paths  available  at  the  completion  of  an  ACTD  consist  of:  termination, 
return  to  technology  base  for  further  development,  residual  utilization, 
initiate  acquisition  at  MS  B  with  major  improvements,  initiate  acquisition 
at  MS  C  with  minor  improvements,  initiate  acquisition  at  FOC  as  COTS 
or  NDI.  The  last  of  these  is  the  desired  goal  of  the  ACTD  process  (South, 
2003). 
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III.  ACTD  TRANSITION  PROCESS  VS  KNOWN  PROBLEMS 


A.  BACKGROUND 

Before  we  can  understand  the  importance  of  planning  for  technology  transition, 
we  must  first  understand  what  technology  transition  means.  Technology  transition  is  the 
process  by  which  technology  deemed  to  be  of  significant  use  to  the  operational  military 
community  is  transitioned  from  the  science  and  technology  environment  to  a  military 
operational  field  unit  for  evaluation  and  then  incorporated  into  an  existing  acquisition 
program  or  identified  as  the  subject  matter  for  a  new  acquisition  program  (Dobbins, 
2004). 


There  are  multiple  challenges  that  are  involved  in  the  transitioning  an  ACTD  into 
the  nonnal  acquisition  process.  Some  of  the  issues  that  continue  to  surface  throughout  all 
the  programs  are: 

1.  Contracting  strategy  this  deals  with  getting  the  best  deal  from  the 
contractor  without  losing  momentum  during  the  transition. 

2.  Interoperability-is  the  ACTD  compatible  with  other  systems? 

3.  Supportability-can  it  be  supported  in  a  cost  effective  manor? 

4.  Test  and  Evaluation-getting  these  people  involved  as  early  as  possible  and 
keeping  them  involved  throughout  the  transition  is  very  important  to  the 
success  of  the  ACTD 

5.  Affordability-assessing  life  cycle  affordability  and  application  of  a  Cost  as 
an  Independent  Variable  (CAIV)  strategy  to  continuously  look  for  ways  to 
reduce  cost. 

6.  Funding-finding  the  right  strategy  to  get  the  resources  for  normal 
acquisition. 

7.  Requirements-establishing  a  mission  need  and  goals  at  the  start  and  ending 
with  system  perfonnance  that  captures  the  technology  maturity  and  the 
knowledge  gained  by  the  war  fighter  in  a  realistic  exercise. 

8.  Acquisition  Program  Documentation-defining  and  planning  for  the 
documentation  required  prior  to  the  acquisition  decision  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004). 
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Being  able  to  address  each  one  of  these  challenges  will  increase  the  probability  of 
ACTDs  transitioning  into  the  acquisition  model.  Even  though  addressing  all  of  these 
challenges  does  not  guarantee  a  successful  transition.  The  options  for  a  failed  program 
are  to  use  it  as  an  interim  capability  as  is,  continue  development,  or  tenninate  the 
program.  Having  a  solid  plan  in  place  way  before  the  transition  phase  will  help  in  the 
success  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  & 
Concepts,  August  2004). 

Although  the  formal  acquisition  process  is  the  primary  means  for  procuring  a  new 
military  capability  the  ACTD  process  is  a  pre-acquisition  action  that  allows  the  war 
lighter  to  assess  its  military  utility  before  going  into  full  production.  The  transition  will 
depend  on  the  Military  Unit  Assessment  (MU A)  that  an  operational  user  will  conduct. 
They  will  concentrate  on  the  if  the  program  is  a  valid  requirement,  that  the  maturity  is 
sufficient  for  the  purpose,  and  the  ease  of  integration  into  a  field  usable  product  (Office 
of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August 
2004).  The  ACTD  Process  is  a  pre-acquisition  activity  that  is  recognized  by  the 
acquisition  system.  Since  FY95,  there  have  been  167  ACTD  programs.  These  programs 
are  presented  in  Table  2.  To  date,  112  have  completed  the  demonstration  phase  (due  to 
the  nature  of  ACTDs,  those  initiated  FY06  or  later  are  typically  still  underway).  Of  those 
completed,  16  have  been  terminated  with  another  3  placed  on  hold.  There  are  74 
programs  that  are  reported  as  transitioned  into  the  acquisition  process  while  19  are  still  in 
the  transition  phase.  Out  of  all  the  years  (excluding  FY06-07),  5 1 .4%  have  been  coded  as 
"transition  on  record"  with  an  additional  13.2%  in  the  transition  phase.  This  is  a 
combined  total  of  64.6%  of  the  programs  that  are  eligible  (established  for  2  or  more 
years)  to  enter  into  the  transition  phase.  Although  these  numbers  do  not  seem  impressive 
the  process  is  making  progress.  A  previous  thesis  project  was  done  by  Matthew  South  in 
2003.  He  reported  that  a  total  of  33%  of  the  programs  had  entered  into  the  transition 
phase  at  the  end  of  FY02.  This  means  that  the  success  rate  has  doubled  in  the  last  5 
years.  For  a  fairly  new  program  it  seems  to  have  the  potential  to  continue  on  this  path 
and  reach  over  80%  success  rate. 
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FY 

INITIATED 

DEMO 

COMPLETE 

IN 

DEMO 

TERMINATED 
PRIOR  MUA 

TERMINATED 
AFTER  MUA 

HOLD 

TRANSITION 
ON  RECORD 

IN 

TRANSITION 

95 

11 

11 

1 

1 

9 

96 

12 

12 

2 

10 

97 

9 

9 

8 

1 

98 

14 

14 

1 

2 

11 

99 

11 

11 

11 

00 

12 

11 

1 

1 

2 

8 

01 

15 

15 

1 

1 

6 

7 

■SB 

18 

15 

3 

2 

1 

7 

5 

mm 

14 

7 

7 

1 

1 

2 

3 

mm 

13 

4 

9 

1 

0 

3 

■SB 

15 

3 

12 

1 

2 

0 

■SB 

13 

0 

13 

0 

0 

07 

10 

0 

10 

0 

0 

Total 

167 

112 

55 

8 

8 

3 

74 

19 

Table  3.  ACTD  Execution  History  (From:  Carson  Excel  Spreadsheet,  2007) 


B.  ACTD  CLASSES 

ACTDs  are  categorized  by  three  classes  or  categories.  They  are  listed  as 
Software/workstation/connno,  Weapons,  sensors,  or  C4ISR  systems,  and  System  of 
systems.  These  are  generic  classes  of  ACTDs  that  present  significantly  different 
transition  challenges. 

1.  Class  I  ACTD 

Software/workstation/commo  (Class  I  ACTDs)  is  typically  information  systems 
with  special  purpose  software  operation  on  commercial  workstations.  They  frequently 
are  required  in  small  quantities  and  that  requirement  can  be  satisfied  without  further 
development  or  production  using  the  residual  ACTD  systems  or  a  few  additional  systems. 
Class  I  is  typically  the  easiest  class  to  manage  from  a  transition  perspective  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004b). 

2.  Class  II  ACTD 

Weapons,  sensors,  or  C4ISR  systems  (Class  II)  are  weapon  or  sensor  systems 
similar  in  concept  to  systems  that  are  acquired  through  the  formal  acquisition  process.  In 
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many  cases  a  Class  II  ACTD  will  be  planned  to  transition  LRIP  (post  MS  C)  following 
the  ACTD,  but  there  may  be  cases  where  it  is  appropriate  to  plan  for  additional 
development  following  the  ACTD  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Advanced  Systems  &  Concepts,  August  2004b). 

3.  Class  III  ACTD 

System  of  systems  (Class  III)  are  an  individual  element  within  the  overall 
architecture  of  a  Class  III  ACTD  maybe  a  fielded  system,  a  system  already  in  acquisition, 
or  a  system  emerging  from  the  technology  base.  The  overall  ACTD  may  involve 
multiple  Program  Executive  Officers,  and  perhaps  multiple  Military  Departments.  The 
challenge  may  therefore  be  to  integrate  and  coordinate  the  individual  transitions  to 
achieve  the  capability  represented  in  the  ACTD.  Although  existing  ACTDs  fit  into  each 
of  the  three  classes  described  above,  the  only  ones  which  have  progressed  to  the  point 
that  a  significant  amount  of  transition  planning  effort  has  been  perfonned  are  in  Class  II 
(Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004b). 

After  completion,  ACTDs  have  two  possible  exit  paths  -  acquisition  or  non¬ 
acquisition.  For  the  non-acquisition  path  the  ACTD  can  be  either  tenninated  due  to  a 
lack  of  military  utility,  shelved  for  further  development,  or  fielded  to  establish  a  residual 
operational  capacity  (South,  2003).  Formal  acquisition  is  based  on  the  level  of 
technology  maturity  demonstrated  and  MUA  success.  Based  on  these  elements  the 
acquisition  can  begin  during  System  Development  and  Demonstration  (SD&D), 
Production  and  Deployment  (P&D),  or  additional  elements  can  be  procured  for 
Operations  and  Support  (O&S)  (South,  2003). 

C.  TRANSITION  STRATEGY 

Technology  transition  into  acquisition  requires  planning  beyond  that  required  for 
initial  technology  development.  Acquisition  programs  involve  a  significant  level  of 
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oversight,  planning  and  milestone  reviews  (Dobbins,  2004).  Although  this  statement 
refers  to  the  nonnal  acquisition  process,  an  ACTD  program  should  not  be  treated  any 
differently  if  the  program  is  expected  to  transition. 

It  is  important  that  the  transition  into  acquisition  occur  smoothly  and  without 
undue  loss  of  momentum.  It  is  critical  that  the  objective  be  identified  at  the  approval  of 
the  ACTD,  and  the  transition  strategy  occurs  during  the  early  stages  of  the  planning 
process  which  should  be  identified  in  the  management  plan  (Office  of  the  Deputy  Under 
Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004).  As  in  most 
military  transactions  a  poor  plan  will  result  in  preventable  issues  to  surface.  These  types 
of  problems  can  usually  be  prevented  when  a  solid,  detailed  plan  is  set  in  place.  Starting 
the  transition  strategy  late  in  the  game  will  most  likely  cause  the  loss  of  momentum  and 
possibly  prevent  the  program  from  entering  the  nonnal  acquisition  process. 

1.  Transition  Plan 

Although  there  is  no  policy  or  other  requirement  specifying  the  contents  of  an 
ACTD  transition  plan,  it  should  address  elements  specific  to  the  technology  being 
transitioned  and  how  the  technology  will  merge  into  the  acquisition  process  of  an  existing 
program.  The  transition  plan  should  be  an  element  of  the  overall  ACTD  management 
plan  and  should  reflect  the  transition  strategy.  It  should  address  the  transition  issues  and 
elements  relevant  to  the  specific  technology  being  transitioned,  including  planning  for 
operational  user  evaluations  (Dobbins,  2004).  Transition  planning  is  fairly  straight 
forward  but  not  at  all  easy.  At  the  beginning  of  the  ACTD,  estimate  whether  the  program 
will  enter  into  the  formal  acquisition  process.  If  entry  is  necessary  the  point  at  which  the 
program  will  enter  needs  to  be  identified  (MS  B,  MS  C).  After  that  a  whole  set  of 
strategies  need  to  be  defined  including  contracting,  supportability,  interoperability, 
affordability,  and  requirements  that  will  support  the  intended  point  of  entry  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004). 
Another  important  piece  of  the  puzzle  is  the  implementation  timelines  for  each  of  the 
above  strategies. 
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a.  Transition  Integrated  Product  Team  (TIPT) 

Part  of  an  effective  transition  plan  and  management  process  is  the 
formation-often  by  the  ACTD  demonstration  manager-and  activation  of  the  transition 
integrated  product  team.  The  TIPT  provides  the  most  natural  means  for  bringing  the  key 
stakeholders  together  to  review  strategies.  It  also  serves  as  a  bridge  between  the  initial 
ACTD  management  planning  activity  and  the  transition  decisions,  assists  in  identifying 
and  resolving  transition  issues.  The  receiving  acquisition  program  office  and  the 
contractors  should  be  included  in  the  TIPT  (Dobbins,  2004). 

b.  Overarching  Integrated  Product  Team  (OIPT) 

As  the  time  for  completion  of  the  ACTD  approaches,  an  overarching 
integrated  product  team  (OIPT)  should  be  formed  as  a  successor  to  the  TIPT.  The  OIPT 
completes  the  remainder  of  the  transition  reviews  (cost,  schedule,  and  performance)  in 
preparation  for  transition  to  acquisition.  The  OIPT  ensures  that  all  of  the  necessary 
elements  and  documentation  are  in  place  for  the  ACTD  to  transition  into  the  acquisition 
program  at  the  appropriate  point  in  the  acquisition  life  cycle.  The  OIPT  will  also  prepare 
for  a  formal  program  review  by  the  defense  acquisition  executive. 

It  is  also  advisable  (not  required)  to  conduct  a  major  review  with  the  Lead 
Service  organization  that  will  be  accepting  both  the  interim  capability  assets  from  the 
ACTD  and  the  objective  system.  This  review  should  occur  at  least  six  months  prior  to 
the  end  of  the  ACTD  and  should  address  the  status  of  preparations  for  operational  support 
(Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004).  Figure  4  shows  how  the  transition  strategy  should  flow  throughout  the 
process. 


40 


ORD 


ACTD 

approval 


Mgnr  pi  an 


Detailed  ACTD 
Planning 

ACTD 

x 


Define  transit  cn  strategy1 

-Acquisition 

■  Procurement 

■  Sup  portability 

■  Interoperability 
-Affordability 
■Requirements  dev 

Utility  Assessment  Plan 

•  Key  perf  parameters 
■MOEs  and  MOP3 
■ACTD  exercises 

■  OPFOR 

■  Simulation 


T 


ik 


TIPT 


■  Review  transition  strategy 

■  Esta Wish  acquisition  B/L 

■  LCC  estimates 

■  Finding  strategy 

■  0  Apr  ogress 

■  Sup  portability 
■Interoperability 

■  Utility  assessment 

■  Operational  preparations 
■Requiremertsdevelopment 

■  PSI  program 


OPT 


■  Independent 
assessm  ent 

•  Resolve  Issues 

•  Ran  next  phase 

•  Finding  strategy 

■  Prepare  for 
program  review 


User  review 

•Opni  concepts 

•  Requirements 

•  Training 

•  Sup  portability 


Figure  4.  Transition  Preparations  (From:  Carson  PPT,  2006) 


2.  Contracting  Strategy 

When  a  technology  transitions  into  acquisition,  there  will  be  some  form  of 
contracting  activity  involved.  The  technology  may  be  inserted  into  an  existing  contract 
whose  terms  and  conditions  will  apply  to  the  new  technology.  The  program  may  enter 
acquisition  as  a  major  upgrade  to  and  existing  system  and  may  require  a  separate 
contracting  effort  or  it  could  enter  acquisition  as  a  major  upgrade  to  an  existing  system 
that  requires  a  separate  contract.  Another  option  is  to  have  the  project  enter  under  a  new 
contract  effort  (Dobbins,  2004). 

The  initial  contracting  strategy  should  be  based  on  the  issues  involved  with  a 
particular  ACTD.  It  should  consider  the  effort  to  be  performed  during  the  ACTD,  as  well 
the  post-ACTD  objective.  A  large  part  of  the  contracting  strategy  is  deciding  where  the 
post-ACTD  will  enter  into  the  acquisition  process.  For  example,  if  the  post-ACTD 
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objective  is  to  enter  directly  into  Low  Rate  Initial  Production  (LRIP),  the  contracting 
strategy  should  tailor  the  plan  to  enter  production  with  the  current  design  but  allow  for 
further  development  after  the  completion  (Office  of  the  Deputy  Under  Secretary  of 
Defense  for  Advanced  Systems  &  Concepts,  August  2004).  This  portion  should  include 
how  DoD  will  procure  additional  units  of  the  ACTD  if  that  is  the  decision  at  the 
conclusion  of  the  phase.  There  are  many  decisions  that  need  to  be  made  within  the 
contracting  strategy.  It  has  become  obvious  over  the  years  that  the  early  these  decisions 
are  made  the  chances  of  major  problems  later  are  reduced. 

The  transition  can  be  easier  if  the  OIPT  works  with  the  acquisition  community  to 
ensure  that  a  proper  contracting  strategy  is  in  place.  The  contracting  strategy  must  also 
make  sense  for  the  particular  technology  along  with  the  existing  acquisition  process. 
There  are  many  different  contracting  strategies,  having  a  liaison  with  the  acquisition 
manager  for  the  program  into  which  the  technology  will  transition  will  significantly  ease 
the  transition  process  (Dobbins,  2004).  In  the  end  the  contracting  strategy  alternatives, 
subsequent  to  the  ACTD  contract,  must  be  specified  in  the  solicitation.  The  possibility  of 
continuing  with  the  ACTD  contractor  into  production  should  be  clearly  communicated  to 
potential  offers.  DoD  should  be  forthcoming  within  certain  parameters  of  uncertainties 
that  exist. 


3.  Transition  Funding 

RDT&E  funding  for  ACTDs  can  currently  be  planned,  programmed,  and 
budgeted  through  two  sources: 

1 .  Military  Departments/Agencies 

2.  OSD  can  supplement  the  service/agency  to  cover  cost  in  three  areas: 

a.  added  costs  incurred  when  the  technology  programs  are  reoriented 
to  support  the  ACTD. 

b.  costs  due  to  any  requirement  to  provide  additional  quantities  of 
hardware. 

c.  cost  for  technical  support  for  two  year  of  field  operations  following 
the  ACTD. 
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However,  funding  to  support  the  follow-on  activity  (development,  LRIP,  full  rate 
production,  or  purchase  of  additional  quantities  of  commercial  items)  is  not  typically 
funded  by  OSD  or  the  Service/Agency  until  the  ACTD  demonstrates  military  utility. 
This  lack  of  prior  funding  creates  a  significant  challenge  that  must  be  addressed  as  a  part 
of  the  transition  effort  (Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced 
Systems  &  Concepts,  August  2004).  There  are  three  follow-on  funding  strategies  that  are 
currently  recognized: 

1.  High  Military  Utility-No  Resources  programmed.  When  an  ACTD  has 
significant  military  capability  but  no  resources  have  been  provided  to 
support  the  program.  The  lead  service  and  present  to  the  Defense 
Resource  Board  (DRB)  a  funding  request.  This  will  interrupt  on-going 
programs  funding  by  taking  money  away  from  another  program  in  order  to 
fund  the  ACTD.  This  type  of  funding  strategy  should  only  be  used  in  an 
"urgency  of  need"  situation  that  requires  rapid  acquisition  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004). 

2.  Military  Utility  Established-No  Resources  Programmed.  Lead  service 
waits  until  the  end  of  the  ACTD  to  request  funds.  Since  the  POM  is  a  2 
year  process  the  money  is  not  available  until  2  years  after  the  completion 
of  the  ACTD.  This  means  that  the  continuity  from  an  ACTD  to  an 
acquisition  program  may  be  broken,  and  momentum  lost  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts, 
August  2004). 

3.  Assume  Success  For  Some  ACTDs-Program  Resources  In  Anticipation  of 
Follow-On  Acquisition.  One  way  to  avoid  a  break  in  the  continuity 
between  an  ACTD  and  the  follow-on  acquisition  program  is  to  establish  a 
budget  line  with  funding,  dedicated  solely  to  the  acquisition  of  the  ACTD. 
This  is  a  high  risk  move  because  if  the  ACTD  does  not  prove  military 
utility  and  is  cancelled  then  the  money  set  aside  can  not  be  used  on 
anything  else  and  would  be  lost.  This  would  be  normally  done  when 
military  utility  is  expected  to  be  high,  and  where  there  are  early 
indications  that  the  expectations  will  be  met.  If  for  some  reason  the 
program  becomes  joint,  the  lead  service  can  transfer  the  resources  to  the 
Joint  Lead  Service  for  execution  (Office  of  the  Deputy  Under  Secretary  of 
Defense  for  Advanced  Systems  &  Concepts,  August  2004).  This  strategy 
will  prevent  a  break  in  continuity  altogether. 
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D.  ASSOCIATED  RISK 


The  overarching  mantra  of  the  ACTD  process  is  increased  rapidity  in  the  delivery 
of  relevant  technology  to  the  war  fighter.  In  order  to  accomplish  this,  the  procedures  and 
regulations  that  have  grown  around  formal  acquisition  programs  have  to  be  re-examined 
to  determine  bottlenecks  within  the  process.  One  of  the  prevalent  arguments  against  the 
formal  acquisition  process  is  the  formality  of  the  acquisition  process  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  2007).  The 
belief  exists  that  this  degree  of  formality  precipitates  an  unnecessary  degree  of  oversight 
that  bogs  the  process  down.  This  fact,  however,  may  not  be  as  detrimental  as  it  appears 
on  the  surface. 

In  1998,  the  Congressional  Budget  Office  (CBO)  issued  a  report  on  ACTD 
programs  (Congressional  Budget  Office,  1998).  Within  the  report,  the  CBO  dug  below 
the  surface  issue  of  apparent  superfluous  oversight  and  identified  one  of  the  subcutaneous 
(and  unforeseen)  effects  of  stripping  back  control  — it  introduces  further  risks  into  the 
ACTD  process.  ACTD  programs,  by  their  nature,  are  conceived  with  inherent  technical 
risk  but  lack  of  proper  management  introduces  a  new  subset  of  risk. 

1.  Questionable  Project  Selections 

One  of  the  harshest  criticisms  leveled  against  ACTD  programs  in  the  report 
(Congressional  Budget  Office,  1998)  was  in  regards  to  the  selection  process.  The 
overarching  concern  with  this  risk  was  that  an  improper  selection  of  an  ACTD  that  was 
not  as  technologically  viable  as  another  alternative  would  become  more  difficult  to 
manage.  Because  ACTDs  fall  outside  the  realm  of  traditional  acquisition  processes,  the 
ability  to  mitigate  risk  through  more  structured  reviews  is  degraded  within  an  ACTD 
program.  This  inability  to  grasp  and  reel  in  technological  difficulties  has  the  potential  to 
translate  into  significant  cost  overruns. 
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2.  Ambiguous  Criteria  in  the  Selection  Process 


The  risk  drawn  out  by  the  CBO  (Congressional  Budget  Office,  1998)  is  not  one 
that  berates  the  selection  process  of  ACTDs  per  se;  instead,  it  addresses  how  ACTD 
program  offices  interpret  two  constraints  bounding  the  selection  process.  The  first 
boundary  drawn  out  is  the  degree  of  technical  maturity.  The  main  critique  of  the 
technological  maturity  is  a  precise  definition  of  what  it  means  for  a  program  to  be 
sufficiently  mature.  GAO  notes  that  their  interpretation  of  maturity  within  DoD  is  a 
system  that  has  demonstrated  successes  at  the  subsystem  or  component  level  (United 
States  General  Accounting  Office,  2002).  They  do  not  denote  what  constitutes  a 
successful  performance.  The  lack  of  fidelity  within  the  definition  has  led  to  the  adoption 
of  many  definitions  of  technology  maturity  within  ACTD  processes.  DoD  has  tried  to 
mitigate  this  situation  by  stating  that  ACTD  programs  be  assessed  on  their  maturity  with 
the  same  TRLs  used  by  traditional  acquisition  programs  (United  States  General 
Accounting  Office,  2002).  Another  weakness  noted  is  the  seeming  lack  of  understanding 
on  when  to  apply  the  notion  of  maturity  to  an  ACTD  program  (Congressional  Budget 
Office,  1998).  The  issue  stems  from  a  propensity  for  program  offices  to  look  toward  the 
ultimate  product  that  an  ACTD  program  provides  to  the  user;  however,  this  lack  of  focus 
on  present  program  maturity  creates  a  disparity  between  determining  if  a  current 
technology  is  sufficiently  mature  and  designing  a  path  to  an  end  product.  In  other  words, 
a  lack  of  understanding  on  the  degree  of  maturity  on  a  current  program  precludes  a  clear 
roadmap  because  program  offices  focus  a  great  deal  of  effort  on  the  end  product. 

The  second  parameter  noted  by  the  CBO  (Congressional  Budget  Office,  1998)  is 
a  disparity  of  ACTD  development  and  assurance  that  they  are  addressing  an  urgent 
requirement.  The  issue  brought  to  light  is  that,  though  the  JROC  provides  initial 
screening  of  candidate  programs  for  the  ACTD  process  they  are  not  active  participants  in 
the  transition  process  (United  States  General  Accounting  Office,  2002).  This  lack  of 
uncertainty  on  the  joint  utility  of  a  program  may  be  one  of  the  factors  prohibiting  ACTDs 
from  transitioning  into  the  formal  acquisition  phase.  Because  the  JROC  does  not  have  a 
role  in  the  transition  process  of  ACTDs,  these  programs  often  lose  upper-level  support. 
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Therefore,  the  transition  process  falls  back  to  the  services  which  often  do  not  support  the 
ACTD  program  because  there  are  other  priority  programs  or  the  ACTD  extends  beyond 
their  primary  mission. 

As  identified  by  the  Deputy  Under  Secretary  of  Defense  for  Advanced  systems 
and  Concepts  (DUSD(AS&C)),  which  is  supposed  to  be  used  by  ACTD  managers  and 
users,  the  following  list  of  criteria  are  used  to  detennine  approval/implementation  of  a 
purposed  ACTD  program: 


Criteria  Indicators  of  Success 

1 .  The  ACTD  met  a  priority  military 
need.  (C-l) 

a.  The  proposed  solution  incorporated  intense  user  involvement  to  evaluate  the  ability  to 
meet  military  needs 

b.  ACTDs  users  had  realistic  and  extensive  military  exercise  opportunities  to  evaluate 
utility  and  gain  experience  with  capabilities 

c  Users  refined  their  operational  requirements,  developed  CONOPS.  and  developed  a 
sound  understanding  of  the  military  utility. 

2  The  ACTD  was  adequately  mature 
<C-2) 

a  ACTD  was  a  mature  or  nearly  mature  technology  based  on  Technology  Readiness 

Level  (TRL)  scale  (minimum  level  5  or  above). 

b.  ACTD  maturity  reduced  the  time  and  risks  associated  with  the  demonstration. 

c.  ACTD  activities  focused  on  integration  and  demonstration  activities  not  technology 
development 

3  Thf  ACTD  d? monstmtons  f 

exercises  were  a  dequa  If  lo  provid  e 
a  unlit. in  utility  assessment  (Co) 

*  Adequate  quantities  of  ACTD  were  procured  «o  provide  a  valid  assessment  of  ns 
capabilities. 

b  ACTD  demom.li  aliou  was  sued  and  stmcluied  to  piovide  a  clear  evaluation  of  military 
capability. 

c.  ACTD  integrated  /  executed  both  developmental  and  operational  T&E  swiftly  and 

economics Uy  to  ensure  that  requirements  were  met  and  the  system  w  as  operationally 

satisfactory  and  ■useful. 

d  The  llset  defined  the  MOEs  and  MOP*  tliat  allowed,  effectiveness  and  suitability  to  be 

characterized. 

e.  User  planned  the  operational  exercises,  typically  including  red  and  blue  forces. 

4  The  ACTD  developer 

demonstrated  the  essential  ACTD 
criteria  (C-4) 

a.  The  potential  or  projected  effectiveness  was  sufficient  to  warrant  consideration  as  an 
ACTD. 

b  The  available  capability  addicssed  a  need  for  which  there  was  no  suitable  solution 

c.  The  ability  of  the  technology*  to  be  interoperable  with  other  systems  on  the  battlefield 
was  verified 

d.  The  fielded  system  would  maintain  a  high  state  of  readiness  and  safety',  using  trained 
operators  and  fliamr&iners.  and  do  so  economically  and  with  the  smallest  possible 
logistical  footprint 

5.  The  ACTD  lead  service  executed 
the  program  appropriately  (C-5) 

a.  The  lead  service-  agency  ensured  the  necessary  planning  for  transition  to  formal 
acquisition  was  accomplished 

b.  The  lead  service- agency  ensured  tiansition  of  the  residual  assets  to  the  user 
organization  and  for  all  aspects  of  then  support. 

6.  ACTD  sponsorship  was  executed 

appropriately  (C-6) 

a.  The  JROC  recommendation  for  lead  service  user  sponsor  was  accepted  bv 

DUS  D(AS&C). 

b  The  user  sponsor  w'as  a  Unified  Commander  (general  rule,  not  as  a  requirement), 
c.  Affordability  goals  were  set  for  acquisition  and  lifc-cyclc  costs  that  permitted  CATV 

trade-offs  and  taler  design-lo-cost  (DTC)  tradeoffs 

7.  The  ACTD  execution  window 
completed  within  two  to  four 
years.  (C-7) 

a.  The  ACTD  completed  all  activities  within  the  expected  tune  (2-4  yrs). 
b  No  significant  schedule  oi  configuration  changes  wete  lequiied  to  meet  objectives 

8  Die  risks  a  ssoeiated  w  ith  the 

ACTD  were  appropriately 
identified  (C-8) 

a.  Risks  were  identified  and  accepted  by  the  primary  stakeholders  of  the  ACTD 
b  Programmatic  risks  (e.g.  cost  and  schedule)  aud  the  operational  risks  related  to  the 
acceptability*  of  the  operational  concepts  necessary  to  realize  the  full  benefit  of  the 
proposed  capability  were  mi  nimized 

c.  System  complexity  (low  to  high)  was  related  to  nsk.  level  (low  to  high),  respectively 
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Criteria 

Indicators  of  Success 

9.  ACTD  funding  was  sufficient  to 
meet  program  requirements  (C-9) 

a.  A  budget  waa  developed  and  vubtintteil  av  a  pait  of  die  proposed  ACTD 
b  The  proper  drategy  was  chosen  for  ©braining  the  resources  necessary  for  acquiring  the 
technology 

c.  AH  costs  associated  with  the  design  and  development  of  the  prototype  system  was 

identified  {this  includes  all  additional  units  required  in  the  ACTD,  all  exercises  that 

must  be  paid  for  by  the  project  and  test  support  costs  including  any  modeling 
simulation  and  analys  is  needed  to  support  the  utility  assessment) 
d  The  ACTD  budget  included  transition  costs  related  to  the  planning  and  pt  epata  lions  for 
acquisition,  as  well  as  the  cost  to  provide  technical  support  for  the  first  two  years  of 
fielding  the  residuals 

10  The  ACTD  executed  ils  transition, 
plans,  to  initiate  acquisition.  (C-10) 

a  The  ACTD  did  not  lose  momentum  in  transitioning  to  the  acquisition  process 
(assuming  the  user  made  a  positive  determination  of  military  utility), 
b  A  clear  acquisition  goal  was  set  for  the  post  ACTD  phases 

c.  Requirements  were  evolved  from  mission  need  and  performance  goals  to  formal 
operational  requirements  documents:  interoperability  documents:  system  performance 
specifications;  and  total  ownership  costs  (manning:,  training  and  sustainability  related 
to  applying  the  technology)  estimates. 

d.  A  contracting  strategy  was  established  that  motivated  the  contractor  to  preside  a  best- 
value  solution  (in  terms  of  overall  life-cycle  cost-effectiveness)  and  permitted 
transition  into  procurement  without  the  loss  of  momentum. 

Table  4.  ACTD  Criteria  and  Indicators  of  Success  (From:  ACTD  Guidelines,  August 

2004) 


3.  Too  Little  Oversight 

One  of  the  risks  identified  by  the  CBO  (Congressional  Budget  Office,  1998)  was 
a  concern  that  the  ACTD  may  be  circumventing  the  acquisition  process  and  not 
supplementing  it.  Restrictions  are  in  place  that  restricts  the  number  of  prototypes  bought 
only  to  those  necessary  for  operational  testing  (Mol,  1998);  however,  ambiguity 
surrounds  what  is  necessary  for  operational  testing  and  for  how  long.  The  concern  is  that 
an  ill-defined  operational  testing  environment  could  involve  extensive  field  trials  with  an 
undo  number  of  test  vehicles. 

In  addition  to  a  proper  of  mix  of  test  vehicles,  Congressional  concerns  also 
surround  the  degree  of  stringency  applied  to  ACTD  contracting  practices  (Congressional 
Budget  Office,  1998).  Again,  due  to  their  operations  outside  the  traditional  acquisition 
process,  ACTDs  are  open  for  interpretation  in  regards  to  contracting  procedures.  There 
appears  to  be  no  fast  procedures  (only  guidelines)  for  the  application  of  contracting 
criteria  when  dealing  with  ACTDs. 
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4. 


Future  Demands  for  Defense  Spending 


The  final  problem  noted  by  the  CBO  (Congressional  Budget  Office,  1998)  is  that 
due  to  their  less-  fonnalized  nature,  ACTDs  often  do  not  have  the  necessary  cost 
projections  available  if  an  ACTD  transitions  into  the  formal  acquisition  process.  This  is  a 
direct  outgrowth  of  the  uncertain  nature  of  ACTDs  in  their  initial  development.  An 
ACTD  lives  in  a  constant  state  of  tennination  or  reversion  back  to  a  lab  environment, 
therefore,  it  is  not  imperative  that  long-range  financial  forecasts  are  formalized  in  the 
early  stages  of  the  program. 

If  an  ACTD  makes  it  past  the  “halfway  point”  in  the  ACTD  process,  the  guideline 
is  for  the  transition  integrated  product  team  (IPT)  to  begin  developing  costs  associated 
with  acquiring  the  program.  Long-term  plans,  however,  regarding  ACTDs  are  inherently 
uncertain  and  little  certainty  can  be  assigned  to  cost  analysis  data  generated  in  the  long¬ 
term  for  the  program.  This  uncertainty  in  costing  sets  up  a  scenario  where  ACTDs  are 
funded  with  a  chunk  mentality  (Congressional  Budget  Office,  1998)  which  may  or  may 
not  prove  sufficient  to  properly  develop  the  program. 

This  lack  of  adherence  to  formalized  fiscal  planning  processes  may  be  one  of  the 
factors  jeopardizing  the  unwillingness  to  fund  an  ACTD’s  transition  (United  States 
General  Accounting  Office,  2002).  The  main  issue  is  that  the  lack  of  authoritative  cost 
projects  does  not  allow  for  inclusion  in  the  formalized  budgeting  process.  What  is  often 
the  case  is  that  services  must  now  take  funds  from  their  own  coffers  to  support  the 
transition  of  the  program.  As  was  previously  mentioned,  the  lack  of  support  generated 
for  these  programs  by  the  services  can  prevent  programs  from  receiving  funding  for 
transition. 

E.  CHAPTER  SUMMARY 

This  chapter  has  addressed  the  transition  process  of  the  ACTD  programs, 
including  the  challenges  that  each  program  faces  each  time  they  enter  into  the  process. 
The  execution  history  has  identified  all  the  programs  to  date  and  where  they  stand  in  the 
process.  This  helps  identify  where  the  successes  and  weaknesses  are  located  in  the 
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process.  We  discussed  the  different  ACTD  classes  along  with  which  class  is  the  most 
likely  to  proceed  through  the  programs  with  the  least  amount  of  challenges.  Identifying 
the  importance  of  the  transition  strategy  is  a  key  element.  Without  the  early  planning  of 
this  strategy  the  likelihood  of  a  program  entering  into  the  normal  acquisition  process  is 
very  limited.  How  the  TIPT  and  OIPT  play  key  roles  through  out  the  process  but  more 
importantly  in  the  contracting  strategy  phase.  Funding  is  probably  one  of  the  most 
important  phases  of  the  transition.  Without  the  proper  funding  in  place  all  the  planning  is 
irrelevant.  Finally,  we  reviewed  known  problems  with  the  ACTD  process  from  project 
selection,  ambiguous  criteria,  down  to  the  funding  issues  involved  with  the  process. 


49 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


50 


IV.  ACTD  ANALYSIS 


This  chapter  will  describe  the  approach  employed  for  an  overall  analysis  of  those 
ACTD  programs  for  which  data  was  provided. 

The  initial  section  will  demonstrate  the  methods  to  both  summarize  and  codify  the 
data  contained  with  the  ACTD  management  plans.  The  data  gleaned  from  the  program 
management  plans  was  not  standardized  and  had  to  be  encapsulated  and  encoded  for 
effective  analysis.  To  understand  this,  the  approach  to  analyze  the  encoded  data  will  be 
explained. 

Once  an  understanding  of  the  data  and  encoding  techniques  are  established,  a 
brief  overview  of  the  statistical  model  employed  will  be  discussed. 

Finally,  given  the  outputs  from  the  statistical  model,  possible  causal  relationships 
will  be  examined. 

A.  DATA  SUMMARY 

In  our  analysis  we  limited  our  investigation  to  one  quadrant  of  the  two-by-two 
diagram  (upper  right)  below  (Table  5).  The  quadrant  we  investigated  involved 
information  provided  within  the  management  plan  (vertical  axis)  and  a  hypothesized 
effect  (horizontal  axis)  from  the  given  data  set.  Others  could  look  at  this  by  using  a 
different  source  but  the  same  variable  (lower  right),  using  a  different  source  different 
variable  (lower  left),  or  same  source  different  variable  (upper  left). 

The  initial  challenge  when  presented  with  the  provided  ACTD  data  was  the 
application  of  a  systematic  approach  to  summarize  the  data.  The  data  we  examined 
pertaining  to  the  ACTD  programs  was  the  program  management  plans.  Though  the 
management  plans  contained  information  pertaining  to  a  myriad  of  aspects  of  the  ACTD 
programs,  they  were  not  uniform  in  their  approach  or  depth.  Therefore,  the  approach  to 
data  summarization  was  to  identity  all  aspects  of  the  ACTD  management  plans  that  were 
a)  available  for  review  and  b)  might  possibly  affect  the  final  disposition  of  the  program. 
Two  important  limitations  to  the  study  should  therefore  be  mentioned  here.  First,  the 
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data  source  we  used  is  a  limitation  in  the  sense  that  we  were  limited  to  coding  what  data 
was  available  (or  missing)  across  management  plans.  Second,  we  used  our  own 
hypotheses  about  which  factors  might  possibly  affect  transition,  and  were  guided  by  our 
advisors  knowledge  of  prior  work  on  technology  transition.  What  emerged  from  this 
coding  process  was  a  set  of  coded  variables  that  we  believe  is  reasonable  and  robust. 

In  order  to  ensure  uniformity,  each  management  plan  was  reviewed  for  those 
program  facets  that  were  identified  within  the  document.  Once  a  comprehensive  list  of 
readily  available  program  attributes  were  delineated  from  the  collection  of  management 
plan  scrubs,  a  crosscheck  identified  those  attributes  which  were  common  throughout  the 
entirety  of  the  plans. 

When  the  listing  of  common  attributes  was  finally  compiled,  it  was  reviewed  for 
possible  significance.  The  information  was  subdivided  into  two  categories.  One 
category  was  information  that  was  available  for  all  programs  but  was  reckoned 
inconsequential  to  program  outcome.  The  second  category  was  information  that  was 
available  for  all  programs  and  was  reckoned  consequential  to  program  success  or  failure. 
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Table  5.  2x2  Diagram  (From:  Wideman/Phelps,  2007) 
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From  the  list  of  substantial  data  shared  by  all  programs,  eleven  questions  were 
constructed  to  identify  those  areas  and  ancillary  aspects  of  those  areas  for  analysis. 
Before  the  questions  could  be  put  to  the  management  plans,  however,  a  system  of  coding 
the  data  needed  to  be  established. 

B.  CODING 

The  coding  applied  to  the  data  (and  serving  as  the  basis  for  analysis)  fell  into 
three  distinct  categories. 

The  first  category  was  binary.  If  the  question  lent  itself  to  a  simple  yes  or  no 
response,  this  approach  to  coding  was  used. 

The  second  category  was  a  scaling  system.  This  approach  provided  an  approach 
to  coding  those  questions  that  were  more  subjective  in  nature.  Not  surprisingly  this 
approach  constituted  a  large  portion  of  the  questions.  Underlying  the  scaling  process  was 
an  attempt  to  create  a  consistent  and  objective  evaluation  of  variables.  Our  intuition  is 
that  program  management  provides  a  consistently  optimistic  forecast  for  program  success 
no  matter  the  underlying  difficulties  inherent  to  the  program.  Scaling  the  responses 
allowed  the  data  presented  in  the  management  plan  to  strengthen  program  management’s 
hopeful  outlook  against  a  more  objective  approach. 

The  final  category  was  number  entry.  This  approach  was  utilized  in  those 
instances  where  actual  numbers  were  either  provided  or  could  be  determined  by  some 
rudimentary  analysis,  and  the  two  aforementioned  methods  were  not  appropriate  for 
coding. 

We  examined  the  following  variables. 

1.  Transition,  or  Not 

This  was  the  dependent  variable  in  our  analysis  of  the  thirty-eight  programs.  Of 
the  programs  studied  19  of  them  transitioned  into  the  acquisition  process  while  the  other 
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19  did  not.  A  binary  method  was  employed  to  mark  each  ACTD  as  transitioning  or  not. 
A  "1"  was  assigned  to  those  programs  that  transitioned  and  a  "0"  to  the  non-transitioned 
ACTDs. 

We  coded  the  following  independent  variables. 

2.  Manager  Assignment 

Per  ACTD  Guidelines:  Management  Plans  (Office  of  the  Deputy  Under  Secretary 
of  Defense  for  Advanced  Systems  &  Concepts,  2007),  by  the  point  that  a  management 
plan  is  produced  for  a  given  ACTD,  the  guidance  states  that  managers  of  the  project  must 
be  identified.  The  extent  of  management  identified  is  not  explicit  but  implies  all 
anticipated  management  will  be  named. 

Seven  key  members  of  program  management  (IPT  manager,  oversight  group, 
executive  agent,  user  sponsor,  technical  manger,  transition  manager,  and  operations 
manager)  were  selected  for  identification.  Once  the  positions  were  detennined,  the 
management  plans  were  reviewed  for  name  identification.  If  the  plan  provided  a  specific 
point  of  contact,  the  corresponding  position  was  assigned  a  code  of  “1,”  marking  it  as 
having  met  identification  criteria.  When  the  plan  was  reviewed  and  specific  members 
were  identified,  the  management  members  were  summed  to  present  a  total  number  of 
those  members  actually  assigned  out  of  7. 

3.  Budget  Matched  to  Schedule 

In  order  to  determine  the  correspondence  between  schedule  and  forecasted 
funding  data,  the  two  provided  schedules  were  compared  for  discrepancies.  A 
rudimentary  exam  of  the  two  schedules  often  noted  minor  discrepancies  that  may  have 
negated  this  metric;  however,  further  scrutiny  identified  finesse  in  budgeting  and 
development  scheduling  that  required  investigation.  Many  of  these  programs  were 
overlapping  the  provided  funding  by  extending  the  program  schedule  past  a  noted 
funding  timeline.  A  closer  examination  noted  that  many  of  these  efforts  were  coming 
from  further  auxiliary  funding  sources  or  were  play  in  the  schedule  for  further 
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development.  Though  the  charted  timeline  provided  an  initial  conflict  at  first  glance,  the 
written  narrative  delineating  the  development  effort  often  alleviated  such  disparities. 

Given  this  information,  we  coded  plans  with  a  binary  code  (i.e.  0/1)  depending  on 
whether  there  was  a  reasonable  match  between  the  development  schedule  (both  written 
and  charted)  against  the  funding  matrices  provided  in  the  management  plan. 

4.  Established  Military  Need 

The  determination  of  an  established  and  justified  military  need  necessitated  the 
use  of  a  scaling  system.  When  the  management  plans  were  compared  across  one  another 
it  became  apparent  that  a  binary  coding  system  was  wholly  inadequate  in  measuring  this 
program  facet.  The  elaboration  and  citations  contained  within  this  portion  of  the 
management  plan  lent  itself  to  scaleable  grading  system. 

At  the  lowest  end  of  the  scale  was  a  low  rating.  The  low  rating  reflected  either  a 
complete  lack  of  inclusion  of  the  military  need  justification  or  a  generic  justification  that 
had  a  few  specific  connections  to  the  particular  ACTD  program.  Typically  a  need  was 
posited  based  on  a  strategic  or  tactical  deficiency,  but  little  or  no  effort  was  applied  to 
demonstrating  how  the  ACTD  contributed  to  fulfilling  the  claimed  need. 

The  middle  rating  was  a  medium  rating.  A  medium  rating  demonstrated  an  effort 
to  tie  the  ACTD  program  to  a  particular  military  need;  however,  it  did  not  demonstrate 
conclusively  how  the  particular  program  would  help  remedy  the  situation.  Within  this 
rating,  a  continual  tendency  of  the  management  plans  was  to  generalize  the  military  need 
and  hypothesize  how  the  ACTD  could  be  molded  to  meet  this  need. 

We  coded  plans  "high"  when  the  management  plans  identified  specific  military 
failings  that  facilitated  the  overarching  military  need  and  explicitly  addressed  how  the 
particular  ACTD  in  development  would  fulfill  this  need  and  showed  a  clear  path  on  how 
to  achieve  this  aim. 
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5.  Technology  Maturity 


One  of  the  most  difficult  program  parameters  to  gauge  was  the  level  of 
technology  maturity  of  the  ACTD.  The  difficulty  arose  primarily  from  the  degree  (or 
lack  thereof)  of  data  provided  within  the  management  plans  regarding  technology 
maturity.  Traditionally,  the  maturity  of  technology  is  spelled  out  in  a  regulated  system 
known  as  Technology  Readiness  Levels  (TRL)  (Advanced  Systems  &  Concepts,  2006). 
However,  the  management  plans  provided  were  inconsistent  in  their  mention  of  TRL’s 
pertaining  to  their  programs  or  did  not  provide  enough  technical  data  to  ascertain  a  rough 
estimate  on  where  the  program  may  fall  on  the  scale.  Another  method  of  scaling  was 
needed. 

The  foundation  of  most  of  the  ACTD’s  relied  heavily  on  the  success  of  software 
development.  The  thread  of  software  success  ran  through  all  of  the  management  plans 
and  provided  a  basis  for  some  scaling  of  technology  maturity.  In  lieu  of  TRL’s,  an 
alternative  system  for  gauging  technology  maturity,  known  as  ImpACT,  was  utilized 
(Advanced  Systems  &  Concepts,  2006).  ImpACT  deals  specifically  with  software 
development  and  the  degree  of  impact  of  commercial-off-the-shelf  (COTS)  software  on 
the  success  of  the  development  effort.  ImpACT  allowed  for  an  effective  scaling 
methodology  to  be  developed  based  upon  the  description  of  development. 

The  highest  rating  was  an  all  COTS/pre-existing  rating.  Given  the  lack  of  detail 
within  the  preponderance  of  management  plans,  this  rating  reflected  an  assignment  to 
software  (technology)  that  they  had  been  developed  and  tested  and  demonstrated  to  be  a 
viable  option.  It  does  not  reflect  the  anticipated  degree  of  success  of  militarizing  the 
COTS  but  simply  shows  that  the  technology  is  one  in  existence  with  some  degree  of 
practicality. 

The  middle  rating  was  partial  COTS  rating.  This  rating  demonstrated  that  at  least 
part  of  the  software  (technology)  development  effort  was  more  mature  than  new 
development  due  to  the  inclusion  of  COTS.  It  also  took  into  account  that  because  the 
effort  was  partially  composed  of  new  development  (in  addition  to  COTS)  the  technology 
maturity  was  lessened. 


56 


The  lowest  rating  was  the  new  rating.  Reflected  within  this  rating  is  the 
knowledge  that  the  software  (technology)  has  no  aspect  that  has  either  been  fully 
developed  or  tested. 

6.  Transition  Strategy 

The  transition  strategy  was  also  coded  using  a  scaling  metric.  Like  the 
aforementioned  military  need  criteria,  transition  strategy’s  scale  had  to  be  constructed, 
not  against  an  available  benchmark,  but  against  the  verbiage  provided  within  the 
management  plans  themselves. 

A  low  rating  reflected  either  a  complete  omission  of  a  transition  strategy  or  an 
acknowledgment  that  it  was  the  hope  of  the  program  office  that  the  program  would 
transition  at  a  later  date. 

The  medium  rating  was  given  to  those  ACTD  efforts  which  had  a  transition  plan 
in  development.  In  other  words,  the  program  management  plan  had  either  a  detailed  plan 
to  develop  a  transition  plan  or  had  a  transition  plan  lacking  significant  detail. 

The  highest  rating  was  given  to  transition  plans  that  had  exacting  details  (e.g., 
involved  parties,  sequential  processing,  timelines,  etc.).  This  rating  reflected  a  transition 
plan  that  marked  a  clear  path  to  incorporate  the  ACTD  beyond  the  development  phase. 

7.  Timeline  Requirement 

Per  ACTD  Guidelines:  Formulation,  Selection  and  Initiation  (Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  &  Concepts,  August  2004), 
ACTD  programs  are  intended  to  have  a  development  and  demonstration  lifetime  of 
between  two  and  four  years.  This  timeline  underscores  the  primary  presupposition  for 
ACTD  programs  to  be  rapid  development  efforts  to  get  critical  technologies  to 
combatants. 

For  the  supplied  management  plans,  the  development  and  demonstration 
schedules  were  reviewed  for  number  of  years.  The  years  were  counted  and  actual 
numbers  were  entered.  The  numbers  entered  do  not  reflect  ancillary  efforts  that  may 


57 


have  preceded  the  ACTD.  For  consistency,  if  such  efforts  were  noted  on  the  schedule, 
we  discounted  them  and  did  not  include  them  in  the  overall  count.  Given  our  sample  size 
(over  30)  and  random  selection  of  our  sample,  it  is  reasonable  to  assume  that  we  have 
randomized  for  ancillary  efforts,  and  that  these  should  not  bias  our  statistical  analysis  in 
any  particular  way. 

8.  Plan  Depth 

Plan  depth  was  coded  with  actual  page  counts  of  the  management  plans.  The 
assumption  here  is  that  increased  page  counts  are  indicators  of  a  more  developed 
management  plans. 

9.  Number  of  Parties  Involved 

For  the  number  of  parties  involved,  actual  counts  of  the  parties  were  used.  There 
were  two  considerations  made  in  ascertaining  party  involvement  numbers. 

The  first  consideration  is  that  one  mention  reflected  one  count.  Many  of  the 
involved  parties  were  mentioned  under  numerous  development  phases,  as  well  as, 
numerous  integrated  product  team  (IPT)  membership.  Second  we  considered  how  ACTD 
management  is  intended  for  implementation.  The  program  is  subdivided  into  sub-efforts 
(e.g.,  IPT).  Each  sub-effort  is  managed  by  an  overarching  agency  appointed  by  ACTD 
program  management.  The  purpose  of  the  underlying  management  is  to  align  and  rectify 
concerns  within  their  functional  area. 

It  might  be  noted  that  the  number  of  parties  involved  is  an  indicator  of  the 
coordination  complexity  involved  in  a  particular  ACTD  program.  More  complex 
coordination  arrangements  may  be  reasoned  to  be  more  prone  to  breakdown,  and 
therefore  to  failure  for  a  program  to  transition. 

10.  Capital  Investment  Requirement 

This  metric  is  summation  of  budgeted  dollars  laid  out  in  the  management  plan. 
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11.  Technology  Complexity  and  Risk  Assessment 


These  last  two  metrics  were  coded  separately.  However,  they  are  intrinsically 
related  and  will  be  discussed  together.  Technology  complexity  was  not  a  metric  included 
in  the  original  management  plans.  It  was  a  subjective  measurement  we  included  in  our 
consideration  of  the  overall  complexity  of  the  program.  Each  was  coded  as  high, 
medium,  or  low. 

First  we  coded  technology  complexity. 

1.  We  looked  at  the  scale  of  proven  technologies  versus  unproven 
technologies  within  the  program. 

2.  We  then  verified  whether  or  not  a  clear  plan  of  technical  integration  was 
present.  A  clear  plan  was  considered  one  that  addressed  an  obvious  path 
for  integrating  the  technologies  as  a  whole  to  accomplish  the  function  of 
the  program. 

3.  Finally,  if  the  management  plan  mentioned  utilization  of  COTS  but  also 
noted  a  lack  of  forecasted  integration  between  COTS  programs,  we  coded 
it  as  complex.  We  assigned  a  subjective  rating  of  high,  medium,  or  low 
based  upon  these  three  factors  taken  as  a  whole. 

Second,  we  coded  plan  risk. 

1 .  Management  plans  that  identified  risk  in  a  program  usually  identified  it  as 
low-to-medium  risk.  Not  all  management  plans  included  their  own  risk 
assessment  and,  given  the  criticality  of  a  risk  analysis  in  a  development 
program,  we  included  a  subjective  risk  analysis.  We  coded  these  and 
included  them  in  our  regression  analysis. 

2.  We  coded  plans  based  on  specific  mentions  of  risk  analysis  and  risk 
mitigation  efforts  or  procedures  that  were  being  put  in  place.  We 
considered  information  contained  in  the  management  plans  on  risk 
mitigation  efforts  in  place  or  being  put  in  place  as  part  of  the  management 
plan  for  the  particular  ACTD  in  question. 

3.  We  then  coded  the  management  plans  based  on  our  own  assessment  of 
their  risk  level.  In  part,  the  risk  measure  we  used  reflects  our  assessment 
of  the  nine  aforementioned  metrics.  This  metric  was  utilized  to  provide  an 
independent  and  (relatively)  objective  assessment  outside  of  program 
management  authorship  of  the  overall  program  risk,  given  the  data 
provided  in  the  management  plan. 


59 


c. 


METHODOLOGY 


After  we  coded  the  variables  we  analyzed  them  using  multivariate  regression  so 
as  to  examine  their  relationship  with  ACTD  transition. 

1.  Multivariate  Regression  Analysis-Explanation 

Multivariate  regression  can  establish  that  a  set  of  independent  variables  explains  a 
proportion  of  the  variance  in  a  dependent  variable  at  a  significant  level  (through  a 
significance  test  of  R~),  and  can  establish  the  relative  predictive  importance  of  the 
independent  variables  (by  comparing  beta  weights).  Power  terms  can  be  added  as 
independent  variables  to  explore  curvilinear  effects.  Cross-product  terms  can  be  added  as 
independent  variables  to  explore  interaction  effects.  One  can  test  the  significance  of 
difference  of  two  R“'s  to  detennine  if  adding  an  independent  variable  to  the  model  helps 
significantly.  Using  hierarchical  regression,  one  can  see  how  most  variance  in  the 
dependent  can  be  explained  by  one  or  a  set  of  new  independent  variables,  over  and  above 
that  explained  by  an  earlier  set.  Of  course,  the  estimates  (b  coefficients  and  constant)  can 
be  used  to  construct  a  prediction  equation  and  generate  predicted  scores  on  a  variable  for 
further  analysis  (Garson,  2007). 

Dummy  variables  are  a  way  of  adding  the  values  of  a  nominal  or  ordinal  variable 

to  a  regression  equation.  The  standard  approach  to  modeling  categorical  variables  is  to 

include  the  categorical  variables  in  the  regression  equation  by  converting  each  level  of 

each  categorical  variable  into  a  variable  of  its  own,  usually  coded  0  or  1.  For  instance,  the 

categorical  variable  "region"  may  be  converted  into  dummy  variables  such  as  "East," 

"West,"  "North,"  or  "South."  Typically  "1"  means  the  attribute  of  interest  is  present  (ex., 

South  =  1  means  the  case  is  from  the  region  South).  Of  course,  once  the  conversion  is 

made,  if  we  know  a  case's  value  on  all  the  levels  of  a  categorical  variable  except  one,  that 

last  one  is  determined.  Why  not  run  separate  regressions?  It  is  true  that  one  approach  to  a 

categorical  variable  in  regression  would  be  to  run  separate  regressions  for  each  category. 

While  this  is  feasible  for  a  single  variable  such  as  gender,  running  a  male  and  a  female 

regression,  it  is  not  the  best  approach  for  two  reasons.  In  practical  terms,  if  there  are 

multiple  categorical  variables  each  with  multiple  categories  (levels),  the  number  of 
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needed  regressions  may  become  unwieldy.  In  statistical  tenns,  we  will  lose  power  since 
each  regression  will  have  a  smaller  sample  size  than  if  there  were  one  overall  regression. 
That  is,  we  will  be  more  likely  to  make  Type  II  errors  (false  negatives,  thinking  there  is 
no  relationship  when  in  reality  there  is)  (Garson,  2007). 
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Table  6.  Raw  Data  Coding  (From:  Wideman/Phelps  Analysis,  2007) 
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D.  ANALYSIS 


1.  Overall  Model  Results 

The  full  model  (Table  7)  indicates  a  coefficient  of  detennination  adjusted  for 
degrees  of  freedom  (Adjusted  R2)  as  .40.  This  would  suggest  that  40%  of  the  model  is 
explained  by  the  1 1  independent  variables.  When  the  core  model  (Table  8)  was  run  with 
the  three  statistically  significant  variables  (from  the  initial  full  model),  the  adjusted  R2 
climbed  to  .50.  This  indicates  that  the  model  with  these  three  independent  variables 
explains  50%  of  the  variance.  The  output  of  the  regression  model  noted  several  distinct 
areas  with  noticeable  impact  on  the  resolution  of  an  ACTD  program. 


SUMMARY  OUTPUT 


Regression  Statistics 

Multiple  R 

0.76467 

R  Square 

0.58472 

Adjusted  R  Square 

0.40902 

Standard  Error 

0.38953 

Observations 

38 

ANOVA 


df 

SS 

MS 

F 

Significance  F 

Regression 

11 

5.55483 

0.50498 

3.32802 

0.00571 

Residual 

26 

3.94517 

0.15174 

Total 

37 

9.5 

Coefficients 

Standard  Error 

tStat 

P -value 

Lower  95% 

Upper  95% 

Lower  95.0% 

Upper  95.0% 

Intercept 

0.99265 

0.63874 

1.55407 

0.13226 

-0.32031 

2.30560 

-0.32031 

2.30560 

Budget  Matches  Schedule 

0.72701 

0.19680 

3.69424 

0.00103 

0.32249 

1.13153 

0.32249 

1.13153 

Technology  Maturity 

-0.38282 

0.17502 

-2.18728 

0.03791 

-0.74258 

-0.02306 

-0.74258 

-0.02306 

Risk  Assessment 

-0.29174 

0.17528 

-1 .66446 

0.10803 

-0.65202 

0.06855 

-0.65202 

0.06855 

Capital  Investment 

0.00041 

0.00048 

0.85332 

0.40127 

-0.00057 

0.00138 

-0.00057 

0.00138 

T ransition  Strategy 

0.06007 

0.11339 

0.52980 

0.60074 

-0.17300 

0.29315 

-0.17300 

0.29315 

Military  Need  Established 

0.07130 

0.13854 

0.51464 

0.61115 

-0.21347 

0.35607 

-0.21347 

0.35607 

Page  Count  (Plan  Depth) 

-0.00289 

0.00627 

-0.46013 

0.64925 

-0.01578 

0.01001 

-0.01578 

0.01001 

2-4  Year  Requirement 

0.06853 

0.15948 

0.42974 

0.67092 

-0.25928 

0.39635 

-0.25928 

0.39635 

Technology  Complexity 

0.05759 

0.14643 

0.39328 

0.69732 

-0.24340 

0.35858 

-0.24340 

0.35858 

Total  Managers  Identified 

-0.01249 

0.05208 

-0.23991 

0.81228 

-0.11954 

0.09455 

-0.11954 

0.09455 

Parties  Involved 

0.01112 

0.05353 

0.20774 

0.83705 

-0.09892 

0.12116 

-0.09892 

0.12116 

Table  7.  Regression  model  for  Significant  Variables  Only  (From:  Wideman/Phelps 

Analysis,  2007) 


Several  authors  have  offered  guidelines  for  the  interpretation  of  a  correlation 
coefficient  (StatSoft,  2003).  As  Cohen  himself  has  observed,  however,  all  such  criteria 
are  in  some  ways  arbitrary  and  should  not  be  observed  too  strictly.  This  is  because  the 
interpretation  of  a  correlation  coefficient  depends  on  the  context  and  purposes.  A 
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correlation  of  0.9  may  be  very  low  if  one  is  verifying  a  physical  law  using  high-quality 
instruments,  but  may  be  regarded  as  very  high  in  the  social  sciences  where  there  may  be  a 
greater  contribution  from  complicating  factors  (StatSoft,  2003). 

2.  Independent  Variables  that  are  Significant  Predictors  of  ACTD 

Transition 

The  output  of  the  regression  model  noted  three  distinct  variables  that  had  an 
impact  on  the  resolution  of  an  ACTD  program. 

We  examined  these  in  a  second  regression  model  -  Figure  6.  This  model  has  an 
overall  adjusted  R2  of  0.50,  which  means  that  these  3  variables  can  be  used  to  generate  a 
simple  model  that  is  a  strong  predictor  of  ACTD  transition  results,  i.e.,  here  is  a  simple, 
parsimonious  set  of  predictors. 

These  areas  are  discussed  below  in  an  attempt  to  garner  an  understanding  of  the 
causal  relationships  that  might  explain  the  statistical  results  we  found. 

a.  Budget  Matched  to  Schedule 

The  first  variable  that  was  statistically  significant  was  “budget  data 
matched  to  schedule.”  Those  programs  that  progressed  successfully  were  more  likely  to 
have  funding  available  to  match  program  scheduling.  Conversely,  those  programs  which 
failed  to  successfully  progress  were  more  likely  to  have  the  opposite  relationship. 

Given  the  importance  of  a  program’s  schedule  and  the  supporting  funding 
means,  proper  alignment  between  the  two  is  essential.  The  data  presented  in  the 
management  plan  stage  of  ACTD  programs  demonstrated  that  this  consideration  was  not 
always  rectified  satisfactorily. 

The  correspondence  between  a  properly  aligned  budget  and  schedule  and 
the  ultimate  success  or  failure  of  an  ACTD  program  is  notable  for  several  possible 
reasons.  The  failure  to  align  two  program  facets  so  inherently  related  demonstrates  an 
absence  of  consideration  for  one  or  the  other  when  constructing  the  profile.  This  lack  of 
communications  could  be  facilitated  through  poorly  aligned  communications  methods  or 
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a  lack  of  congruence  when  developing  the  management  plan.  Overall,  it  portrays  a 
potential  instance  where  these  two  facets  are  developed  with  either  minimal  consideration 
or  concern  for  the  other. 

Given  the  omnipresent  status  of  an  ACTD’s  budget  it  is  hard  to  justify  that 
this  is  as  prevalent  as  the  data  presents.  What  this  relationship  presents  (in  addition  to 
lack  of  consideration)  is  a  degree  of  living  beyond  current  constraints.  A  program 
schedule  contrary  to  funding  profile  projects  a  sense  of  a  project  lacking  the  proper 
understanding  to  align  the  disjointed  facets.  The  technical  schedule  is  held  as  a 
projection  to  achieve  a  viable  program  and  an  inability  to  mold  it  to  funding  constraints 
demonstrates  a  questionable  understanding  of  the  program  as  it  currently  stands. 

b.  Technology  Maturity 

Another  area  cited  with  high  significance  is  the  degree  of  technology 
maturity.  The  more  mature  an  ACTD’s  technology  was  (heavily  cited  as  COTS  in  this 
instance)  the  higher  was  the  probability  of  success  for  the  program. 

The  correlation  between  technology  maturity  and  program  success  is 
important  in  the  context  of  an  ACTD’s  existence.  The  ACTD  program  exists  to  rapidly 
apply  technology  to  a  real-world  concern.  In  other  words,  it  is  about  the  proper 
application  of  technology. 

ACTDs  either  utilize  existing  technologies  in  a  new  manner,  mix  new  and 
existing  technologies  to  augment  current  capabilities,  or  develop  new  technologies.  The 
issue  is  that  simultaneous  development  is  occurring  and  the  more  certainty  (or  proven 
ingredients)  that  can  be  included  in  the  outset,  the  focus  on  the  solution  becomes  more 
salient.  While  the  utilization  of  technology  addresses  the  technical  aspect  of  a  real-world 
military  issue,  there  is  also  the  tactical  concern  of  how  to  employ  said  technology.  The 
more  proven  a  system’s  underlying  technology  is  allows  for  more  rapid  evolution  to  a 
war  fighter's  demands  while  concurrently  not  promising  more  than  can  be  provided. 
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c. 


Risk  Assessment 


The  final  area  with  a  reasonably  high  significance  is  the  risk  assessment. 
Those  programs  coded  as  lower  risk  were  more  likely  to  succeed.  In  many  ways,  this 
corresponds  to  the  abovementioned  conclusion  drawn  from  technology  maturity. 

One  additional  consideration  here  (in  addition  to  technology  maturity)  is 
the  certainty  that  an  overall  risk  assessment  can  be  inferred  from.  When  management 
plans  consistently  they  demonstrate  a  propensity  towards  marking  their  own  programs  as 
having  low  technical  risk  opens  the  forum  for  more  stringent  observation.  Those  plans 
that  have  failed  to  provide  a  thorough  analysis  have  opened  themselves  up  for  the 
possibility  of  termination  or  a  serious  degradation  in  their  support. 


SUMMARY  OUTPUT 


Regression  Statistics 

Multiple  R 

0.73743 

R  Square 

0.54381 

Adjusted  R  Square 

0.50355 

Standard  Error 

0.35702 

Observations 

38 

ANOVA 


df 

ss 

MS  F 

Significance  F 

Regression 

3 

5.16616 

1.72205  13.50994 

0.00001 

Residual 

34 

4.33384 

0.12747 

Total 

37 

9.5 

Coefficients 

Standard  Error  t  Stat 

P-value 

Lower  95% 

Upper  95% 

Lower  95.0%) 

Upper  95.0%o 

Intercept 

1.17339 

0.55039  2.13191 

0.04032 

0.05486 

2.29192 

0.05486 

2.29192 

Budget  Matches  Schedule 

0.77935 

0.14193  5.49106 

0.00000 

0.49091 

1 .06779 

0.49091 

1 .06779 

Technology  Maturity 

-0.36725 

0.14892  -2.46604 

0.01886 

-0.66990 

-0.06460 

-0.66990 

-0.06460 

Risk  Assessment 

-0.21433 

0.13300  -1.61151 

0.11631 

-0.48461 

0.05596 

-0.48461 

0.05596 

Table  8.  Regression  Model  for  Significant  Variables  Only  (From:  Wideman/Phelps 

Analysis,  2007) 


3.  Correlation  between  Significant  Variables  -  Could  we  Simplify  the 
Model  Further  by  Eliminating  Anything  Else? 

Once  the  significant  variables  were  identified  we  looked  at  the  correlation 
between  the  three. 

The  three  variables  that  we  found  to  be  correlated  were  technology  maturity, 

Budget  matches  Schedule,  and  risk  assessment.  These  three  had  a  positive  correlation. 

The  relationship  between  three  variables  is  such  that  as  one  variable's  values  tend  to 
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increase,  then  the  other  variable’s  values  also  tend  to  increase.  This  is  represented  by  a 
positive  correlation  coefficient  (StatSoft,  2003).  We  also  found  that  some  of  the  variables 
had  negative  correlations.  This  means  that  the  relationship  between  two  variables  is  such 
that  as  one  variable's  values  tend  to  increase,  while  the  other  variable's  values  tend  to 
decrease.  If  the  budget  increases  then  the  maturity  level  would  tend  to  decrease  (StatSoft, 
2003). 


Total 

Managers 

Identified 

Budget 

Matches 

Schedule 

Military 

Need 

Established 

Technology 

Maturity 

Transition 

Strategy 

2-4  Year 
Requirement 

Parties 

Involved 

Technology 

Complexity 

Risk 

Assessment 

Page 

Count 

(Plan  Capital 

Depth)  Investment 

Total  Managers  Identified 

1 

Budget  Matches  Schedule 

0.55817 

1 

Military  Need  Established 

0.43561 

0.43503 

1 

Technology  Maturity 

0.32664 

0.41750 

-0.03499 

1 

Transition  Strategy 

0.34858 

0.29011 

0.66989 

-0.05051 

1 

2-4  Year  Requirement 

0.07191 

-0.04942 

-0.20404 

0.35085 

-0.19280 

1 

Parties  Involved 

0.54582 

0.32021 

0.44129 

0.05774 

0.39214 

-0.26816 

1 

Technology  Complexity 

-0.08823 

-0.31208 

0.25035 

-0.52671 

0.17046 

-0.13318 

0.01922 

1 

Risk  Assessment 

-0.35662 

0.47130 

-0.02527 

0.77122 

0.07443 

-0.24736 

-0.10886 

0.67607 

1 

Page  Count  (Plan  Depth) 

0.59889 

0.49521 

0.63518 

0.09658 

0.65794 

-0.12157 

0.46631 

-0.07085 

-0.21105 

1 

Capital  Investment 

0.26217 

0.16029 

0.02544 

0.04517 

0.18892 

-0.20567 

0.13642 

-0.01376 

-0.03373 

0.09782  1 

Figure  5.  Correlation  Table  from  Data  Set  (From:  Wideman/Phelps  Analysis,  2007) 


4.  Non-Significant  Variables 


Non-significant  results  for  variables  are  just  as  important  as  significant  variables. 
There  are  two  particular  variables  that  were  non- significant  that  are  especially  worth 
mentioning.  Though  the  technology  maturity  was  deemed  to  be  significant  to  the 
determination  of  an  ACTD’s  success  prospects,  technology  complexity  did  not.  What 
may  be  underlying  this  phenomenon  is  the  coding  method  chosen.  While  technology 
maturity  lends  itself  to  more  objective  coding,  technology  complexity  is  a  subjective 
measure  applied  in  consideration  of  all  programmatic  features.  Through  an  attempt  to 
summarize  the  program  in  a  single  metric,  the  metric  mirrors  the  program.  By  not 
singularizing  within  the  metric,  the  program’s  success  is  inherently  tied  to  the  metric. 

In  addition  to  the  occurrence  of  technology  complexity  is  an  issue  of  parties 
involved.  The  negligence  of  this  metric  to  play  a  critical  role  in  a  program’s  ultimate 
resolution  is  born  out  of  management  foresight  from  the  ACTD  program  office  (and  is 
hinted  at  in  the  coding  process).  Though  a  multiple  players  may  be  involved  in  a 
program’s  development,  their  potential  disputes  may  have  been  are  filtered  by  the  time 
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the  management  plan  is  in  place,  i.e.,  the  management  plan  may  be  considered  an 
indicator  of  dispute  resolution,  without  which  it  would  have  never  been  completed.  This 
might  explain  the  non-significance  of  this  variable. 
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V.  RECOMMENDATIONS  AND  CONCLUSIONS 


A.  RECOMMENDATIONS 

After  comparing  38  programs  of  which  half  transitioned  into  the  normal 
acquisition  process  we  have  found  that  some  variables  within  the  management  plans  of 
ACTDs  are  powerful  predictors  of  whether  ACTD  programs  will  transition  or  not.  These 
results  suggest  recommendations  to  make  the  ACTD  process  more  successful,  as  follows: 

1 .  Ensure  that  the  over  all  technology  maturity  level  is  high.  This  means  the 
appropriate  interoperability  of  all  the  sub-systems  must  achieve  a  certain 
maturity  level.  Having  COTS  sub-systems  is  a  good  start  but  without 
bringing  them  all  together  to  prove  military  utility  will  be  less  favorable  to 
transition.  Earlier  in  this  paper  we  talked  about  Technology  Readiness 
Levels.  These  seem  to  be  very  important  in  the  transition  process. 
Although  the  management  plans  don’t  specify  exactly  what  TRL  the 
program  enters  into  the  ACTD  our  judgment  was  based  on  how  much  of 
the  program  was  partial  or  all  Commercial  Off  The  Shelf  (COTS).  By 
introducing  a  program  that  is  lower  on  the  TRL  scale  will  most  likely 
decrease  its  chances  of  having  a  successful  transition.  It  is  very  important 
to  obtain  the  highest  maturity  level  when  entering  a  program  into  the 
ACTD  process.  The  more  items  in  a  program  that  are  COTS  the  better 
chance  of  a  transition  will  occur.  A  future  study  of  program  TRLs  is 
recommended  due  to  the  fact  that  we  found  it  to  be  statistically  significant 
in  our  study.  This  would  entail  finding  the  TRL  for  each  and  every  COTS 
item  used  in  a  program  to  determine  the  overall  Technology  Readiness 
Level. 

2.  Adequate  and  coordinated  funding  is  a  must  for  any  program.  As  we 
stated  earlier  the  funding  must  match  the  schedule  in  order  for  the  program 
to  have  a  chance  to  transition.  Having  the  budget  in  place  will  increase 
the  chances  of  a  transition.  This  is  not  anything  new.  We  concentrated  on 
whether  the  budget  actually  matched  the  schedule.  This  seemed  to  have 
an  effect  on  the  outcome  of  the  program.  If  a  program  had  a  3  year  budget 
set  in  place  and  it  was  scheduled  as  a  4  year  process  then  this  will  most 
likely  prevent  the  program  from  transitioning.  By  requiring  the  lead 
service  to  plan  a  budget  around  the  schedule  would  help  the  process.  This 
would  require  the  program  to  initiate  a  request  for  Planning,  Program,  and 
Budgeting  system  funds  associated  with  the  expected  Liscal  Year  of 
transition.  Correct  planning  will  prevent  the  program  from  loosing 
momentum  or  being  stalled  to  the  point  of  no  longer  being  of  value  2  years 
later  when  the  budget  is  approved.  A  closer  look  at  the  budget  compared 
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to  the  schedule  will  be  in  order.  Our  study  shows  that  this  is  another 
statistically  significant  issue  on  whether  a  program  transitions  or  not. 

3.  A  realistic  risk  assessment  must  be  included  within  the  management  plan. 
Including  a  lessened  estimate  of  programmatic  risk  may  make  the 
immediate  plan  more  readily  acceptable  but  does  not  institute  a  stable 
framework  to  further  develop  the  program.  In  addition  to  realism,  the  risk 
assessment  must  also  be  detailed  enough  to  delineate  mitigation  efforts  to 
offset  known  risks.  Mitigation  efforts  must  be  at  a  detail  to  demonstrate  a 
competent  understanding  that  the  risks  facing  a  program  are 
acknowledged  and  understood.  This  may  make  the  initial  plan  harder  to 
accept,  though  it  portrays  a  realistic  understanding  of  the  program  being 
developed.  Risk  assessment  is  important  to  the  transition  process.  Even 
though  a  subjective  analysis  was  used  to  categorize  this  particular  variable 
having  a  solid  risk  mitigation  plan  in  place  will  help  any  program.  By 
making  the  lead  sponsor  put  the  risk  level  in  the  management  plan  will 
force  the  teams  to  understand  where  the  program  stands  and  how  difficult 
the  transition  can  be.  The  programs  that  we  coded  as  high  risk  normally 
did  not  address  risk  in  any  fonn  within  the  management  plan.  A 
recommendation  to  specifically  address  the  actual  risk  the  program 
involves  will  possibly  enhance  the  transition  of  ACTDs. 

We  also  made  another  observation  during  this  study,  which  we  mention  here  for 
completeness.  It  seems  that  the  military  in  general  lives  off  a  Standard  Operating 
Procedure  (SOP)  in  everything  we  do.  Looking  through  multiple  ACTD  management 
plans  the  lack  of  standardization  was  very  obvious.  If  a  standardized  template  along  with 
what  infonnation  is  required  was  put  into  place  it  would  eliminate  submission  of  a  sub¬ 
standard  plan.  Our  observations  in  general  were  the  management  plans  that  incorporated 
every  suggestion  that  is  listed  on  the  ACTD  website  transitioned.  Even  though  many  that 
did  not  follow  these  suggestions  also  transitioned  but  over  all  the  plans  that  transitioned 
addressed  a  majority  of  the  ACTD  suggestions.  On  average  the  programs  that 
transitioned  had  a  page  count  of  29.3  pages  where  the  non-transition  programs  had  14.3 
pages. 

B.  CONCLUSIONS 

The  problem  with  any  acquisition  program  is  the  time  acquired  for 
implementation.  By  changing  an  ACTD  program  today  the  results  will  not  surface  for  a 
minimum  of  two  years.  Even  though  we  would  like  to  be  proactive  in  improving  the 
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process,  it  is  very  hard  when  the  time  line  for  the  results  is  in  the  distant  future.  The  best 
action  to  take  is  make  the  change  and  wait.  The  dilemma  is  by  only  changing  one  thing 
at  a  time  will  take  forever  to  get  the  process  efficient  enough  to  rely  on.  On  the  other 
hand  if  you  change  to  many  things  at  once  you  might  not  be  able  to  determine  the  true 
reason  for  the  change.  One  change  may  be  less  relevant  than  another.  If  you  continue  to 
improve  on  the  lesser  of  the  changes  this  may  actually  give  a  negative  result  in  the  long 
run.  Therefore  improving  the  ACTD  process  may  take  time.  It  seems  that  the  best 
course  of  action  would  be  patience  and  change  one  recommendation  at  a  time  and  see 
what  the  results  bring.  This  method  would  narrow  down  the  most  important  problems 
that  are  keeping  ACTDs  from  transitioning  into  the  nonnal  acquisition  process.  By 
changing  multiple  issues  at  the  same  time  will  prevent  the  largest  problem  from 
surfacing.  This  could  cause  a  much  larger  delay  in  correcting  the  more  important  issues. 

In  May  of  2003  new  acquisition  instructions  were  approved  for  implementation 
following  the  Secretary  of  Defenses  August  2002  cancellation.  Associated  with  these 
processes  are  mechanisms  that  are  designed  to  foster  efficiency  and  innovation  in 
conjunction  with  future  Evolutionary  Acquisition  Strategies.  Evolutionary  acquisition 
strategies  are  the  preferred  approach  to  satisfying  operational  needs.  The  two 
mechanisms  that  have  been  identified  include  incremental  development  and  spiral 
development.  Under  incremental  development  the  end-state  requirement  is  known  and 
will  be  met  over  time  through  several  system  increments,  or  configurations.  Under  Spiral 
development  the  desired  capabilities  are  identified  but  the  end-state 
capabilities/requirements  are  not  specifically  known  at  program  initiation.  Spiral 
development  is  an  iterative  process  that  links  users  to  developers  through  an  approach  of 
continuous  development  and  deployment  of  both  software  and  hardware.  The  end-state 
capabilities/requirements  for  the  future  increments  are  dependent  upon  technology 
maturation  and  user  feedback  from  the  initial  increments.  Of  these  two  mechanisms, 
spiral  development  shall  be  the  preferred  process  (Department  of  Defense,  2003). 

We  have  examined  1 1  variables  to  detennine  their  effect  on  the  outcome  of  an 
ACTD  program.  Of  the  1 1  variables  investigated,  three  proved  to  have  a  measurable 
relationship  to  the  ultimate  determination  of  an  ACTD’s  success.  Budget  Matched  to 
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Schedule,  Technology  Maturity,  and  Risk  Assessment  were  the  three  variables  that 
demonstrated  a  correlation  to  an  ACTD’s  transition.  It  is  our  conclusion  that  when 
supplied  with  management  plans  for  ACTD  programs,  decision  authorities  should  further 
scrutinize  these  areas  as  they  have  an  impact  on  the  life  of  the  program. 
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